View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0002031GNUnetobsoletepublic2011-12-23 22:162024-05-03 13:51
ReporterLRN Assigned ToLRN  
PrioritynormalSeverityminorReproducibilityN/A
Status closedResolutionfixed 
Product Version0.9.0 
Target Version0.9.1Fixed in Version0.9.1 
Summary0002031: [patch] No lowlevel UTF-8 support in GNUnet
Descriptionplibc is not UTF-8 (or even UTF-16) compatible, and neither is GNUnet. It does have a function for UTF-8 conversion, but it's rarely used. Particularly, it's not used in some places in disk.c

GTK uses an OS-dependent encoding for its internal filename representation. On W32 it's UTF-8, on other systems it's locale-depndend or something. Because of that the names that GTK gives us on W32 can not be passed to GNUnet, and names obtained from GNUnet util library can not be passed to GTK (put into entries, treemodels, etc). Not without conversion. And that conversion is rarely made. Also, the code is simply unaware of the difference.

Quick&dirty fix:
Let GNUnet be non-UTF8 (well, it will use FS encoding, which MIGHT be UTF-8 on some OSes).
Set up a border between GTK and GNUnet, and convert everything that crosses it back and forth.

Longterm fix:
Complete UTF-8 support in GNUnet and plibc.
TagsNo tags attached.
Attached Files
0001-Temporary-fix-for-charset-conversion.patch (33,178 bytes)    
From 5c5f49c95a5def2db09459fa9ce92301c3d8d4d6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=A0=D1=83=D1=81=D0=BB=D0=B0=D0=BD=20=D0=98=D0=B6=D0=B1=D1?=
 =?UTF-8?q?=83=D0=BB=D0=B0=D1=82=D0=BE=D0=B2?= <lrn1986@gmail.com>
Date: Sat, 24 Dec 2011 23:07:30 +0400
Subject: [PATCH] Temporary fix for charset conversion

---
 src/fs/gnunet-fs-gtk-common.c                      |   87 ++++++----
 src/fs/gnunet-fs-gtk-common.h                      |   18 ++
 src/fs/gnunet-fs-gtk-download.c                    |   10 +-
 src/fs/gnunet-fs-gtk-edit_publish_dialog.c         |    5 +-
 src/fs/gnunet-fs-gtk-event_handler.c               |   36 ++++-
 src/fs/gnunet-fs-gtk-main_window_adv_pseudonym.c   |    6 +-
 .../gnunet-fs-gtk-main_window_create_pseudonym.c   |   10 +-
 src/fs/gnunet-fs-gtk-main_window_file_publish.c    |   53 ++++++-
 src/fs/gnunet-fs-gtk-main_window_open_directory.c  |   10 +-
 src/fs/gnunet-fs-gtk.c                             |   21 ++-
 src/include/gnunet_gtk.h                           |   28 ++++
 src/lib/nls.c                                      |  167 ++++++++++++++++++++
 12 files changed, 391 insertions(+), 60 deletions(-)

diff --git a/src/fs/gnunet-fs-gtk-common.c b/src/fs/gnunet-fs-gtk-common.c
index b2bf9b2..bd33d17 100644
--- a/src/fs/gnunet-fs-gtk-common.c
+++ b/src/fs/gnunet-fs-gtk-common.c
@@ -26,6 +26,53 @@
 #include "gnunet-fs-gtk-common.h"
 
 /**
+ * Converts metadata specified by @data of size @data_len
+ * and saved in format @format to UTF-8 encoded string.
+ * Works only for C-string and UTF8 metadata formats
+ * (returns NULL for everything else).
+ * Verifies UTF-8 strings.
+ *
+ * @param format format of the @data
+ * @param data data to convert
+ * @param data_len length of the data buffer (in bytes)
+ * @return NULL if can't be converted, allocated string otherwise,
+ *         freeable with GNUNET_free* ().
+ */
+char *
+GNUNET_FS_GTK_dubious_meta_to_utf8 (enum EXTRACTOR_MetaFormat format, const char *data, size_t data_len)
+{
+  gchar *result = NULL;
+
+  if (format == EXTRACTOR_METAFORMAT_UTF8)
+  {
+    /* data must not contain NULLs (hence the -1) */
+    if (g_utf8_validate (data, data_len - 1, NULL))
+      result = GNUNET_strdup (data);
+    else
+    {
+      GNUNET_log (GNUNET_ERROR_TYPE_INFO,
+          "Failed to validate supposedly utf-8 string `%s' of length %u, assuming it to be a C string\n",
+          data, data_len);
+      format = EXTRACTOR_METAFORMAT_C_STRING;
+    }
+  }
+  if (format == EXTRACTOR_METAFORMAT_C_STRING)
+  {
+    if (data_len > 0)
+    { /* There are no guarantees that data is NULL-terminated, AFAIU,
+       * so let's play it safe, shall we?
+       */
+      char *data_copy = GNUNET_malloc (data_len + 1);
+      memcpy (data_copy, data, data_len);
+      data_copy[data_len] = '\0';
+      result = GNUNET_GTK_from_loc_to_utf8 (data_copy);
+      GNUNET_free (data_copy);
+    }
+  }
+  return result;
+}
+
+/**
  * Add meta data to list store.
  *
  * @param cls closure (the GtkListStore)
@@ -49,44 +96,17 @@ GNUNET_FS_GTK_add_meta_data_to_list_store (void *cls, const char *plugin_name,
                                            const char *data, size_t data_len)
 {
   GtkListStore *ls = GTK_LIST_STORE (cls);
-  const gchar *data_to_insert = NULL;
-  gchar *free_data = NULL;
-  gsize rd;
-  gsize wr;
+  gchar *data_to_insert = NULL;
 
-  if (format == EXTRACTOR_METAFORMAT_UTF8)
-  {
-    if (g_utf8_validate (data, data_len, NULL))
-      data_to_insert = data;
-    else
-    {
-      GNUNET_log (GNUNET_ERROR_TYPE_INFO,
-          "Failed to validate supposedly utf-8 string `%s' of length %u, assuming it to be a C string\n",
-          data, data_len);
-      format = EXTRACTOR_METAFORMAT_C_STRING;
-    }
-  }
-  if (format == EXTRACTOR_METAFORMAT_C_STRING)
-  {
-    GError *gerr = NULL;
-    rd = wr = 0;
-    free_data = g_locale_to_utf8 (data, data_len, &rd, &wr, &gerr);
-    if (gerr != NULL)
-    {
-      GNUNET_log (GNUNET_ERROR_TYPE_INFO,
-          "Error when converting a C string `%s' of length %u to utf-8 (converted %u to %u bytes): %s\n",
-          data, data_len, rd, wr, gerr->message);
-      g_error_free (gerr);
-    }
-    data_to_insert = free_data;
-  }
+  data_to_insert = GNUNET_FS_GTK_dubious_meta_to_utf8 (format, data, data_len);
 
   if (NULL != data_to_insert)  
+  {
     gtk_list_store_insert_with_values (ls, NULL, G_MAXINT, 0, type, 1, format,
                                        2, EXTRACTOR_metatype_to_string (type),
-                                       3, data_to_insert, -1);  
-  if (NULL != free_data)
-    g_free (free_data);
+                                       3, data_to_insert, -1);
+    g_free (data_to_insert);
+  }
 
   return 0;
 }
@@ -194,5 +214,4 @@ GNUNET_FS_GTK_mmap_and_scan (const char *filename,
   GNUNET_break (GNUNET_OK == GNUNET_DISK_file_close (fh));
 }
 
-
 /* end of gnunet-fs-gtk-common.c */
diff --git a/src/fs/gnunet-fs-gtk-common.h b/src/fs/gnunet-fs-gtk-common.h
index 9484b0e..d0d7b4f 100644
--- a/src/fs/gnunet-fs-gtk-common.h
+++ b/src/fs/gnunet-fs-gtk-common.h
@@ -101,5 +101,23 @@ GNUNET_FS_GTK_add_meta_data_to_list_store (void *cls, const char *plugin_name,
                                            const char *data_mime_type,
                                            const char *data, size_t data_len);
 
+/**
+ * Converts metadata specified by @data of size @data_len
+ * and saved in format @format to UTF-8 encoded string.
+ * Works only for C-string and UTF8 metadata formats
+ * (returns NULL for everything else).
+ * Verifies UTF-8 strings.
+ *
+ * @param format format of the @data
+ * @param data data to convert
+ * @param data_len length of the data buffer (in bytes)
+ * @return NULL if can't be converted, allocated string otherwise,
+ *         freeable with GNUNET_free* ().
+ */
+char *
+GNUNET_FS_GTK_dubious_meta_to_utf8 (enum EXTRACTOR_MetaFormat format,
+    const char *data, size_t data_len);
+
+
 #endif
 /* end of gnunet-fs-gtk-common.h */
diff --git a/src/fs/gnunet-fs-gtk-download.c b/src/fs/gnunet-fs-gtk-download.c
index 3f5f213..09ae7e1 100644
--- a/src/fs/gnunet-fs-gtk-download.c
+++ b/src/fs/gnunet-fs-gtk-download.c
@@ -73,7 +73,7 @@ GNUNET_GTK_save_as_dialog_delete_event_cb (GtkWidget *widget, GdkEvent  *event,
     return FALSE;
   }
   GNUNET_free_non_null (dc->filename);
-  dc->filename = gtk_file_chooser_get_filename (GTK_FILE_CHOOSER (dlc->dialog));
+  dc->filename = GNUNET_GTK_filechooser_get_filename_loc (GTK_FILE_CHOOSER (dlc->dialog));
   dc->is_recursive =
       (TRUE ==
        gtk_toggle_button_get_active (GTK_TOGGLE_BUTTON (cb))) ? GNUNET_YES :
@@ -165,6 +165,7 @@ GNUNET_FS_GTK_open_download_as_dialog (struct DownloadContext *dc)
   if (dc->filename != NULL)
   {
     char buf[1024];
+    char *buf_utf8;
 
     if (NULL != getcwd (buf, sizeof (buf)))
     {
@@ -173,7 +174,12 @@ GNUNET_FS_GTK_open_download_as_dialog (struct DownloadContext *dc)
         strcat (buf, DIR_SEPARATOR_STR);
         strcat (buf, dc->filename);
       }
-      gtk_file_chooser_set_current_name (GTK_FILE_CHOOSER (ad), buf);
+      buf_utf8 = GNUNET_GTK_from_loc_to_utf8 (buf);
+      if (buf_utf8 != NULL)
+      {
+        gtk_file_chooser_set_current_name (GTK_FILE_CHOOSER (ad), buf_utf8);
+        GNUNET_free (buf_utf8);
+      }
     }
   }
   dlc = g_new0 (struct dialog_context, 1);
diff --git a/src/fs/gnunet-fs-gtk-edit_publish_dialog.c b/src/fs/gnunet-fs-gtk-edit_publish_dialog.c
index 8c0d76b..5324e3a 100644
--- a/src/fs/gnunet-fs-gtk-edit_publish_dialog.c
+++ b/src/fs/gnunet-fs-gtk-edit_publish_dialog.c
@@ -530,6 +530,7 @@ GNUNET_GTK_edit_publication_metadata_preview_file_chooser_button_file_set_cb (
                  (builder,
                   "GNUNET_GTK_edit_publication_metadata_preview_image"));
   gtk_image_set_from_file (image, fn);
+  g_free (fn);
   state->preview_changed = GNUNET_YES;
 }
 
@@ -589,7 +590,7 @@ preserve_meta_items (void *cls, const char *plugin_name,
     GNUNET_break (0);
     return 0;
   }
-
+  /* FIXME: make sure this is all correct UTF-8-wise */
   keep = GNUNET_NO;
   switch (format)
   {
@@ -808,6 +809,7 @@ file_information_update (void *cls, struct GNUNET_FS_FileInformation *fi,
     }
     g_object_unref (finfo);
     g_object_unref (f);
+    g_free (fn);
   }
   GNUNET_CONTAINER_meta_data_destroy (nm);
 
@@ -1016,6 +1018,7 @@ file_information_extract (void *cls, struct GNUNET_FS_FileInformation *fi,
 
 /**
  * Open the dialog to edit file information data.
+ * short_fn MUST be UTF-8-encoded
  */
 void
 GNUNET_FS_GTK_edit_publish_dialog (GtkBuilder *builder, 
diff --git a/src/fs/gnunet-fs-gtk-event_handler.c b/src/fs/gnunet-fs-gtk-event_handler.c
index 23129eb..631c342 100644
--- a/src/fs/gnunet-fs-gtk-event_handler.c
+++ b/src/fs/gnunet-fs-gtk-event_handler.c
@@ -548,6 +548,7 @@ setup_download (struct DownloadEntry *de, struct DownloadEntry *pde,
   GtkTreeIter iter;
   GtkTreePath *path;
   struct SearchResult *srp;
+  gchar *filename_utf8;
 
   if (de == NULL)
   {
@@ -587,13 +588,15 @@ setup_download (struct DownloadEntry *de, struct DownloadEntry *pde,
     return de;
   }
   gtk_tree_path_free (path);
+  filename_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) filename);
   gtk_tree_store_set (de->ts, &iter, 4,
                       (guint) ((size >
                                 0) ? (100 * completed /
                                       size) : 100) /* progress */ ,
-		      6, filename /* filename/description */ ,
+		      6, filename_utf8 /* filename/description */ ,
                       8, "blue" /* status colour: pending */ ,
                       -1);
+  GNUNET_free_non_null (filename_utf8);
   return de;
 }
 
@@ -1111,6 +1114,7 @@ setup_search (struct GNUNET_FS_SearchContext *sc,
   }
   else
   {
+    /* FS_uri functions should produce UTF-8, so let them be */
     if (GNUNET_FS_uri_test_ksk (query))
       tab->query_txt = GNUNET_FS_uri_ksk_to_string_fancy (query);
     else
@@ -1291,6 +1295,16 @@ GNUNET_GTK_add_search_result (struct SearchTab *tab, GtkTreeIter * iter,
                                                      -1);
   if (desc == NULL)
     desc = GNUNET_strdup (_("no description supplied"));
+  else
+  {
+    char *utf8_desc = NULL;
+    utf8_desc = GNUNET_FS_GTK_dubious_meta_to_utf8 (EXTRACTOR_METAFORMAT_UTF8, desc, strlen (desc) + 1);
+    GNUNET_free (desc);
+    if (utf8_desc != NULL)
+      desc = utf8_desc;
+    else
+      desc = NULL;
+  }
   if (uri == NULL)
     uris = GNUNET_strdup (_("no URI"));
   else
@@ -1348,7 +1362,7 @@ GNUNET_GTK_add_search_result (struct SearchTab *tab, GtkTreeIter * iter,
   if (pixbuf != NULL)
     g_object_unref (pixbuf);
   GNUNET_free (uris);
-  GNUNET_free (desc);
+  GNUNET_free_non_null (desc);
   GNUNET_free_non_null (mime);
   tp = gtk_tree_model_get_path (GTK_TREE_MODEL (ts), iter);
   sr->rr = gtk_tree_row_reference_new (GTK_TREE_MODEL (ts), tp);
@@ -1468,6 +1482,16 @@ update_search_result (struct SearchResult *sr,
                                                      -1);
   if (desc == NULL)
     desc = GNUNET_strdup (_("no description supplied"));
+  else
+  {
+    char *utf8_desc = NULL;
+    utf8_desc = GNUNET_FS_GTK_dubious_meta_to_utf8 (EXTRACTOR_METAFORMAT_UTF8, desc, strlen (desc) + 1);
+    GNUNET_free (desc);
+    if (utf8_desc != NULL)
+      desc = utf8_desc;
+    else
+      desc = NULL;
+  }
   mime =
       GNUNET_CONTAINER_meta_data_get_first_by_types (meta,
                                                      EXTRACTOR_METATYPE_MIMETYPE,
@@ -1497,7 +1521,7 @@ update_search_result (struct SearchResult *sr,
                       (gint) availability_rank, -1);
   if (pixbuf != NULL)
     g_object_unref (pixbuf);
-  GNUNET_free (desc);
+  GNUNET_free_non_null (desc);
   GNUNET_free_non_null (mime);
 
   notebook =
@@ -1610,6 +1634,7 @@ setup_publish (struct GNUNET_FS_PublishContext *pc, const char *fn,
   GtkWidget *close_button;
   GtkNotebook *notebook;
   char *size_fancy;
+  char *fn_utf8 = NULL;
 
   if (NULL == parent)
   {
@@ -1676,11 +1701,14 @@ setup_publish (struct GNUNET_FS_PublishContext *pc, const char *fn,
   ent = GNUNET_malloc (sizeof (struct PublishEntry));
   ent->is_top = (parent == NULL) ? GNUNET_YES : GNUNET_NO;
   ent->tab = publish_tab;
-  gtk_tree_store_insert_with_values (publish_tab->ts, &iter, pitrptr, G_MAXINT, 0, fn,
+  fn_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) fn);
+  gtk_tree_store_insert_with_values (publish_tab->ts, &iter, pitrptr, G_MAXINT,
+                                     0, fn_utf8,
                                      1, size_fancy, 2, "white", 3,
                                      (guint) 0 /* progress */ ,
 				     4, ent,
                                      -1);
+  GNUNET_free_non_null (fn_utf8);
   path = gtk_tree_model_get_path (GTK_TREE_MODEL (publish_tab->ts), &iter);
   GNUNET_assert (NULL != path);
   ent->rr = gtk_tree_row_reference_new (GTK_TREE_MODEL (publish_tab->ts), path);
diff --git a/src/fs/gnunet-fs-gtk-main_window_adv_pseudonym.c b/src/fs/gnunet-fs-gtk-main_window_adv_pseudonym.c
index b15060b..711b78f 100644
--- a/src/fs/gnunet-fs-gtk-main_window_adv_pseudonym.c
+++ b/src/fs/gnunet-fs-gtk-main_window_adv_pseudonym.c
@@ -48,12 +48,14 @@ add_to_list (void *cls, const char *name, const GNUNET_HashCode * id)
 {
   GtkListStore *ls = cls;
   GtkTreeIter iter;
+  char *name_utf8;
+  name_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) name);
 
-  gtk_list_store_insert_with_values (ls, &iter, -1, 0, name, 1,
+  gtk_list_store_insert_with_values (ls, &iter, -1, 0, name_utf8, 1,
                                      GNUNET_FS_namespace_create
                                      (GNUNET_FS_GTK_get_fs_handle (), name),
                                      -1);
-
+  GNUNET_free_non_null (name_utf8);
 }
 
 
diff --git a/src/fs/gnunet-fs-gtk-main_window_create_pseudonym.c b/src/fs/gnunet-fs-gtk-main_window_create_pseudonym.c
index e0a8b0e..c912aab 100644
--- a/src/fs/gnunet-fs-gtk-main_window_create_pseudonym.c
+++ b/src/fs/gnunet-fs-gtk-main_window_create_pseudonym.c
@@ -31,6 +31,7 @@ GNUNET_GTK_create_namespace_dialog_response_cb (GtkDialog *dialog,
   gint response_id, gpointer user_data)
 {
   const char *name;
+  gchar *name_loc;
   struct GNUNET_FS_Namespace *ns;
   GtkWidget *ad;
   GtkBuilder *builder;
@@ -51,8 +52,13 @@ GNUNET_GTK_create_namespace_dialog_response_cb (GtkDialog *dialog,
                            (builder,
                             "GNUNET_GTK_create_namespace_name_entry")));
   /* FIXME: show busy dialog while doing key creation */
-  ns = GNUNET_FS_namespace_create (GNUNET_FS_GTK_get_fs_handle (), name);
-  GNUNET_FS_namespace_delete (ns, GNUNET_NO);
+  name_loc = GNUNET_GTK_from_utf8_to_loc ((char *) name);
+  if (NULL != name_loc)
+  {
+    ns = GNUNET_FS_namespace_create (GNUNET_FS_GTK_get_fs_handle (), name_loc);
+    GNUNET_FS_namespace_delete (ns, GNUNET_NO);
+    GNUNET_free (name_loc);
+  }
   gtk_widget_destroy (ad);
   g_object_unref (G_OBJECT (builder));
 }
diff --git a/src/fs/gnunet-fs-gtk-main_window_file_publish.c b/src/fs/gnunet-fs-gtk-main_window_file_publish.c
index 1629842..7172e33 100644
--- a/src/fs/gnunet-fs-gtk-main_window_file_publish.c
+++ b/src/fs/gnunet-fs-gtk-main_window_file_publish.c
@@ -211,6 +211,7 @@ add_file_at_iter (gpointer data, const char *filename, const struct GNUNET_FS_Bl
   GtkTreePath *path;
   uint64_t file_size;
   const char *short_fn;
+  char *short_fn_utf8;
   struct GNUNET_CONTAINER_MetaData *meta;
   struct GNUNET_FS_Uri *ksk_uri;
   GtkTreeStore *ts;
@@ -248,10 +249,11 @@ add_file_at_iter (gpointer data, const char *filename, const struct GNUNET_FS_Bl
   short_fn = filename;
   while (NULL != (ss = strstr (short_fn, DIR_SEPARATOR_STR)))
     short_fn = 1 + ss;
+  short_fn_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) short_fn);
   GNUNET_CONTAINER_meta_data_insert (meta, "<gnunet-gtk>",
                                      EXTRACTOR_METATYPE_FILENAME,
                                      EXTRACTOR_METAFORMAT_UTF8, "text/plain",
-                                     short_fn, strlen (short_fn) + 1);
+                                     short_fn_utf8, strlen (short_fn_utf8) + 1);
   ksk_uri = GNUNET_FS_uri_ksk_create_from_meta_data (meta);
   gtk_tree_store_insert_before (ts, &pos, iter, NULL);
   path = gtk_tree_model_get_path (GTK_TREE_MODEL (ts), &pos);
@@ -268,8 +270,9 @@ add_file_at_iter (gpointer data, const char *filename, const struct GNUNET_FS_Bl
   else
     file_size_fancy = GNUNET_STRINGS_byte_size_fancy (file_size);
   gtk_tree_store_set (ts, &pos, 0, file_size_fancy, 1, (gboolean) do_index, 2,
-                      short_fn, 3, (guint) bo->anonymity_level, 4,
+                      short_fn_utf8, 3, (guint) bo->anonymity_level, 4,
                       (guint) bo->content_priority, 5, fi, -1);
+  GNUNET_free_non_null (short_fn_utf8);
   GNUNET_free (file_size_fancy);
   update_selectivity (data);
 }
@@ -294,6 +297,7 @@ create_dir_at_iter (gpointer data, const char *name, const struct GNUNET_FS_Bloc
   GtkTreeStore *ts;
   GtkBuilder *builder;
   
+  
   builder = GTK_BUILDER (data);
 
   ts = GTK_TREE_STORE (gtk_builder_get_object
@@ -504,6 +508,7 @@ extract_file (struct AddDirContext *adc, const char *filename)
   GNUNET_HashCode hc;
   const char *short_fn;
   const char *ss;
+  char *short_fn_utf8;
 
   adc->dir_entry_count++;
   pd = GNUNET_malloc (sizeof (struct PublishData));
@@ -515,11 +520,12 @@ extract_file (struct AddDirContext *adc, const char *filename)
   short_fn = filename;
   while (NULL != (ss = strstr (short_fn, DIR_SEPARATOR_STR)))
     short_fn = 1 + ss;
+  short_fn_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) short_fn);
   GNUNET_CONTAINER_meta_data_insert (pd->meta, "<gnunet-gtk>",
                                      EXTRACTOR_METATYPE_FILENAME,
                                      EXTRACTOR_METAFORMAT_UTF8, "text/plain",
-                                     short_fn, strlen (short_fn) + 1);
-
+                                     short_fn, strlen (short_fn_utf8) + 1);
+  GNUNET_free_non_null (short_fn_utf8);
 
   gtk_tree_store_insert_before (adc->ts, &pd->iter, adc->parent, NULL);
   GNUNET_CRYPTO_hash (filename, strlen (filename), &hc);
@@ -576,6 +582,7 @@ add_entry_to_ts (GtkTreeStore * ts, GtkTreeIter * iter, const char *filename,
   struct GNUNET_FS_Uri *kill_ksk;
   const char *ss;
   const char *short_fn;
+  char *short_fn_utf8;
   struct stat sbuf;
 
   if (0 != STAT (filename, &sbuf))
@@ -627,9 +634,11 @@ add_entry_to_ts (GtkTreeStore * ts, GtkTreeIter * iter, const char *filename,
   short_fn = filename;
   while (NULL != (ss = strstr (short_fn, DIR_SEPARATOR_STR)))
     short_fn = 1 + ss;
+  short_fn_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) short_fn);
   gtk_tree_store_set (ts, iter, 0, file_size_fancy, 1, (gboolean) do_index, 2,
-                      short_fn, 3, (guint) bo->anonymity_level, 4,
+                      short_fn_utf8, 3, (guint) bo->anonymity_level, 4,
                       (guint) bo->content_priority, 5, fi, -1);
+  GNUNET_free_non_null (short_fn_utf8);
   GNUNET_free (file_size_fancy);
 }
 
@@ -774,8 +783,19 @@ scan_directory (void *cls, const char *filename)
     }
     else
     {
+      char *filename_utf8;
+      const char *ss, *short_fn;
       GNUNET_assert (mcm == NULL);
       /* we're top-level */
+      short_fn = filename;
+      while (NULL != (ss = strstr (short_fn, DIR_SEPARATOR_STR)))
+        short_fn = 1 + ss;
+      filename_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) short_fn);
+      GNUNET_CONTAINER_meta_data_insert (pd->meta, "<gnunet-gtk>",
+                                         EXTRACTOR_METATYPE_FILENAME,
+                                         EXTRACTOR_METAFORMAT_UTF8, "text/plain",
+                                         filename_utf8, strlen (filename_utf8) + 1);
+      GNUNET_free_non_null (filename_utf8);
       add_entry_to_ts (adc->ts, &pd->iter, filename, &adc->bo, adc->do_index,
                        NULL, pd->meta);
     }
@@ -1384,7 +1404,7 @@ GNUNET_GTK_publish_directory_dialog_response_cb (GtkDialog *dialog,
 					   "GNUNET_GTK_publish_directory_dialog"));
   if (response_id == -5)
   {
-    filename = gtk_file_chooser_get_filename (GTK_FILE_CHOOSER (ad));
+    filename = GNUNET_GTK_filechooser_get_filename_loc (GTK_FILE_CHOOSER (ad));
     sb = GTK_SPIN_BUTTON (gtk_builder_get_object (builder,
         "GNUNET_GTK_publish_directory_dialog_expiration_year_spin_button"));
     if (!GNUNET_GTK_get_selected_anonymity_level (builder,
@@ -1539,6 +1559,18 @@ add_updateable_to_ts (void *cls, const char *last_id,
                                                      EXTRACTOR_METATYPE_COMMENT,
                                                      EXTRACTOR_METATYPE_SUBJECT,
                                                      -1);
+  if (desc == NULL)
+    desc = GNUNET_strdup (_("no description supplied"));
+  else
+  {
+    char *utf8_desc = NULL;
+    utf8_desc = GNUNET_FS_GTK_dubious_meta_to_utf8 (EXTRACTOR_METAFORMAT_UTF8, desc, strlen (desc) + 1);
+    GNUNET_free (desc);
+    if (utf8_desc != NULL)
+      desc = utf8_desc;
+    else
+      desc = NULL;
+  }
   gtk_tree_store_insert_with_values (uc->ts, &iter, uc->parent, G_MAXINT, 0,
                                      uc->namespace_name, 1, uc->ns, 2, last_id,
                                      3, GNUNET_FS_uri_dup (last_uri), 4,
@@ -1586,13 +1618,15 @@ add_namespace_to_ts (void *cls, const char *name, const GNUNET_HashCode * id)
   GtkTreeStore *ts = cls;
   struct UpdateableContext uc;
   GtkTreeIter iter;
+  gchar *name_utf8;
 
   uc.parent = &iter;
   uc.namespace_name = name;
   uc.ts = ts;
   uc.ns = GNUNET_FS_namespace_create (GNUNET_FS_GTK_get_fs_handle (), name);
   uc.update_called = GNUNET_NO;
-  gtk_tree_store_insert_with_values (ts, &iter, NULL, G_MAXINT, 0, name, 1,
+  name_utf8 = GNUNET_GTK_from_loc_to_utf8 ((char *) name);
+  gtk_tree_store_insert_with_values (ts, &iter, NULL, G_MAXINT, 0, name_utf8, 1,
                                      uc.ns, 2, NULL /* last-id */ ,
                                      3, NULL /* last-uri */ ,
                                      4, NULL /* meta */ ,
@@ -1601,6 +1635,7 @@ add_namespace_to_ts (void *cls, const char *name, const GNUNET_HashCode * id)
                                      7, TRUE /* update editable */ ,
                                      8, TRUE /* current editable */ ,
                                      -1);
+  GNUNET_free_non_null (name_utf8);
   uc.seen = GNUNET_CONTAINER_multihashmap_create (128);
   GNUNET_FS_namespace_list_updateable (uc.ns, NULL, &add_updateable_to_ts, &uc);
   GNUNET_CONTAINER_multihashmap_destroy (uc.seen);
@@ -1726,6 +1761,7 @@ hide_master_publish_dialog (gpointer user_data, gint ret)
       do
       {
         fi = get_file_information (tm, &iter);
+        /* FIXME: should we convert namespace id and uid from UTF8? */
         GNUNET_FS_publish_start (GNUNET_FS_GTK_get_fs_handle (), fi, namespace,
                                  namespace_id, namespace_uid,
                                  GNUNET_FS_PUBLISH_OPTION_NONE);
@@ -1801,7 +1837,6 @@ GNUNET_GTK_publish_file_dialog_response_cb (GtkDialog *dialog,
   if (response_id == -5)
   {
     /* OK */
-    filename = gtk_file_chooser_get_filename (GTK_FILE_CHOOSER (ad));
     sb = GTK_SPIN_BUTTON (gtk_builder_get_object (builder,
         "GNUNET_GTK_publish_file_dialog_expiration_year_spin_button"));
 
@@ -1816,6 +1851,8 @@ GNUNET_GTK_publish_file_dialog_response_cb (GtkDialog *dialog,
     do_index = gtk_toggle_button_get_active (GTK_TOGGLE_BUTTON (
         gtk_builder_get_object (builder,
         "GNUNET_GTK_publish_file_dialog_do_index_checkbutton")));
+
+    filename = GNUNET_GTK_filechooser_get_filename_loc (GTK_FILE_CHOOSER (ad));
     add_file_at_iter (user_data, filename, &bo, do_index, NULL);
     g_free (filename);
     update_selectivity (user_data);
diff --git a/src/fs/gnunet-fs-gtk-main_window_open_directory.c b/src/fs/gnunet-fs-gtk-main_window_open_directory.c
index 23e71fc..e614451 100644
--- a/src/fs/gnunet-fs-gtk-main_window_open_directory.c
+++ b/src/fs/gnunet-fs-gtk-main_window_open_directory.c
@@ -67,7 +67,7 @@ add_child (void *cls, const char *filename, const struct GNUNET_FS_Uri *uri,
     dmeta = GNUNET_CONTAINER_meta_data_duplicate (meta);
     GNUNET_CONTAINER_meta_data_insert (dmeta, "<user>",
                                        EXTRACTOR_METATYPE_FILENAME,
-                                       EXTRACTOR_METAFORMAT_C_STRING,
+                                       EXTRACTOR_METAFORMAT_UTF8,
                                        "text/plain", acc->filename,
                                        strlen (acc->filename) + 1);
     acc->tab = GNUNET_GTK_add_to_uri_tab (&acc->iter, &acc->par, dmeta, NULL);
@@ -91,7 +91,7 @@ GNUNET_GTK_open_directory_dialog_response_cb (GtkDialog *dialog,
 {
   GtkBuilder *builder;
   GtkWidget *ad;
-  char *filename;
+  char *filename, *filename_utf8;
   struct AddChildContext acc;
 
   builder = GTK_BUILDER (user_data);
@@ -105,12 +105,14 @@ GNUNET_GTK_open_directory_dialog_response_cb (GtkDialog *dialog,
     return;
   }
 
-  filename = gtk_file_chooser_get_filename (GTK_FILE_CHOOSER (ad));
+  filename = GNUNET_GTK_filechooser_get_filename_loc (GTK_FILE_CHOOSER (ad));
+  filename_utf8 = GNUNET_GTK_filechooser_get_filename_utf8 (GTK_FILE_CHOOSER (ad));
   gtk_widget_destroy (ad);
   g_object_unref (G_OBJECT (builder));
-  acc.filename = filename;
+  acc.filename = filename_utf8;
   acc.ts = NULL;
   GNUNET_FS_GTK_mmap_and_scan (filename, &add_child, &acc);
+  g_free (filename_utf8);
   g_free (filename);
 }
 
diff --git a/src/fs/gnunet-fs-gtk.c b/src/fs/gnunet-fs-gtk.c
index 5545371..bf6e171 100644
--- a/src/fs/gnunet-fs-gtk.c
+++ b/src/fs/gnunet-fs-gtk.c
@@ -501,8 +501,11 @@ main_window_search_button_clicked_cb (GtkButton *button, gpointer user_data)
   mime_keyword = NULL; 
   if (mime_model && gtk_combo_box_get_active_iter (mime_combo, &iter))  
     gtk_tree_model_get (mime_model, &iter, 0, &mime_keyword, -1);  
-  if (mime_keyword == NULL)
-    mime_keyword = g_strdup ("");
+  if (strcmp (mime_keyword, " ") == 0)
+  {
+    g_free (mime_keyword);
+    mime_keyword = NULL;
+  }
 
   ref = g_object_get_data (G_OBJECT (toggle_button), "selected-row-reference");
   if (ref)
@@ -516,7 +519,7 @@ main_window_search_button_clicked_cb (GtkButton *button, gpointer user_data)
   query_entry = GTK_ENTRY (gtk_builder_get_object (builder,
 						   "main_window_search_entry"));
   entry_keywords = gtk_entry_get_text (query_entry);
-  keywords = g_strdup_printf ("%s +%s", entry_keywords, mime_keyword);
+  keywords = g_strdup_printf ("%s %s%s", entry_keywords, mime_keyword ? "+" : "", mime_keyword ? mime_keyword : "");
   g_free (mime_keyword);
   if (nsid != NULL)
   {
@@ -613,6 +616,18 @@ add_namespace_to_ts (void *cls, const GNUNET_HashCode * pseudonym,
                                                      EXTRACTOR_METATYPE_SUBJECT,
                                                      EXTRACTOR_METATYPE_KEYWORDS,
                                                      -1);
+  if (description == NULL)
+    description = g_strdup (_("no description supplied"));
+  else
+  {
+    char *utf8_desc = NULL;
+    utf8_desc = GNUNET_FS_GTK_dubious_meta_to_utf8 (EXTRACTOR_METAFORMAT_UTF8, description, strlen (description));
+    GNUNET_free (description);
+    if (utf8_desc != NULL)
+      description = utf8_desc;
+    else
+      description = NULL;
+  }
   gtk_tree_store_insert_with_values (ts, &iter, NULL, G_MAXINT, 0, ns_name, 1,
       nsid, 2, root, 3, description, -1);
   GNUNET_free (ns_name);
diff --git a/src/include/gnunet_gtk.h b/src/include/gnunet_gtk.h
index 4f3e2cd..2ce6447 100644
--- a/src/include/gnunet_gtk.h
+++ b/src/include/gnunet_gtk.h
@@ -117,6 +117,34 @@ GNUNET_GTK_tray_icon_create (GtkWindow * main, const char *icon_name,
 void
 GNUNET_GTK_tray_icon_destroy (void);
 
+char *
+GNUNET_GTK_from_utf8_to_loc (gchar *str_utf8);
+
+char *
+GNUNET_GTK_from_loc_to_utf8 (gchar *str_loc);
+
+char *
+GNUNET_GTK_from_filename_to_utf8 (gchar *filename);
+
+char *
+GNUNET_GTK_from_utf8_to_filename (gchar *str_utf8);
+
+char *
+GNUNET_GTK_from_loc_to_filename (gchar *str_loc);
+
+char *
+GNUNET_GTK_from_filename_to_loc (gchar *filename);
+
+/* Returns filename form filechooser, encoded in locale-dependent
+ * encoding, suitable to be given to CRT and/or GNUnet
+ */
+char *
+GNUNET_GTK_filechooser_get_filename_loc (GtkFileChooser *fc);
+
+gchar *
+GNUNET_GTK_filechooser_get_filename_utf8 (GtkFileChooser *fc);
+
+
 
 /* ******************* main loop ***************** */
 
diff --git a/src/lib/nls.c b/src/lib/nls.c
index a040f8e..f313ce5 100644
--- a/src/lib/nls.c
+++ b/src/lib/nls.c
@@ -53,4 +53,171 @@ GNUNET_GTK_setup_nls ()
 #endif
 }
 
+
+char *
+GNUNET_GTK_from_utf8_to_loc (gchar *str_utf8)
+{
+  char *str_loc;
+  const char *loc_charset;
+  gboolean is_UTF8;
+
+  if (NULL == str_utf8)
+    return NULL;
+
+  is_UTF8 = g_get_charset (&loc_charset);
+  if (is_UTF8)
+    str_loc = GNUNET_strdup (str_utf8);
+  else
+    str_loc = GNUNET_STRINGS_from_utf8 (str_utf8, strlen (str_utf8), loc_charset);
+
+  return str_loc;
+}
+
+char *
+GNUNET_GTK_from_loc_to_utf8 (gchar *str_loc)
+{
+  char *str_utf8;
+  const char *loc_charset;
+  gboolean is_UTF8;
+
+  if (NULL == str_loc)
+    return NULL;
+
+  is_UTF8 = g_get_charset (&loc_charset);
+  if (is_UTF8)
+    str_utf8 = GNUNET_strdup (str_loc);
+  else
+    str_utf8 = GNUNET_STRINGS_to_utf8 (str_loc, strlen (str_loc), loc_charset);
+
+  return str_utf8;
+}
+
+/* This is copied from GLib */
+static gboolean
+get_filename_charset (const gchar **filename_charset)
+{
+  const gchar **charsets;
+  gboolean is_utf8;
+  
+  is_utf8 = g_get_filename_charsets (&charsets);
+
+  if (filename_charset)
+    *filename_charset = charsets[0];
+  
+  return is_utf8;
+}
+
+char *
+GNUNET_GTK_from_filename_to_utf8 (gchar *filename)
+{
+  char *str_utf8;
+  const char *filename_charset;
+  gboolean is_UTF8;
+
+  if (NULL == filename)
+    return NULL;
+
+  is_UTF8 = get_filename_charset (&filename_charset);
+  if (is_UTF8)
+    str_utf8 = GNUNET_strdup (filename);
+  else
+    str_utf8 = GNUNET_STRINGS_to_utf8 (filename, strlen (filename), filename_charset);
+
+  return str_utf8;
+}
+
+char *
+GNUNET_GTK_from_utf8_to_filename (gchar *str_utf8)
+{
+  char *filename;
+  const char *filename_charset;
+  gboolean is_UTF8;
+
+  if (NULL == str_utf8)
+    return NULL;
+
+  is_UTF8 = get_filename_charset (&filename_charset);
+  if (is_UTF8)
+    filename = GNUNET_strdup (str_utf8);
+  else
+    filename = GNUNET_STRINGS_from_utf8 (str_utf8, strlen (str_utf8), filename_charset);
+
+  return filename;
+}
+
+char *
+GNUNET_GTK_from_loc_to_filename (gchar *str_loc)
+{
+  char *filename;
+  const char *filename_charset;
+  const char *loc_charset;
+  gboolean is_filename_UTF8, is_loc_UTF8;
+
+  if (NULL == str_loc)
+    return NULL;
+
+  is_filename_UTF8 = get_filename_charset (&filename_charset);
+  is_loc_UTF8 = g_get_charset (&loc_charset);
+  if (is_filename_UTF8 && is_loc_UTF8)
+    filename = GNUNET_strdup (str_loc);
+  else if (is_filename_UTF8)
+    filename = GNUNET_STRINGS_to_utf8 (str_loc, strlen (str_loc), loc_charset);
+  else if (is_loc_UTF8)
+    filename = GNUNET_STRINGS_from_utf8 (str_loc, strlen (str_loc), filename_charset);
+  else
+    /* Pray that iconv() knows these charsets */
+    filename = GNUNET_STRINGS_conv (str_loc, strlen (str_loc), loc_charset, filename_charset);
+
+  return filename;
+}
+
+char *
+GNUNET_GTK_from_filename_to_loc (gchar *filename)
+{
+  char *str_loc;
+  const char *loc_charset;
+  const char *filename_charset;
+  gboolean is_loc_UTF8, is_filename_UTF8;
+
+  if (NULL == filename)
+    return NULL;
+
+  is_filename_UTF8 = get_filename_charset (&filename_charset);
+  is_loc_UTF8 = g_get_charset (&loc_charset);
+  if (is_loc_UTF8 && is_filename_UTF8)
+    str_loc = GNUNET_strdup (filename);
+  else if (is_loc_UTF8)
+    str_loc = GNUNET_STRINGS_to_utf8 (filename, strlen (filename), filename_charset);
+  else if (is_filename_UTF8)
+    str_loc = GNUNET_STRINGS_from_utf8 (filename, strlen (filename), loc_charset);
+  else
+    /* Pray that iconv() knows these charsets */
+    str_loc = GNUNET_STRINGS_conv (filename, strlen (filename), filename_charset, loc_charset);
+
+  return str_loc;
+}
+
+/* Returns filename form filechooser, encoded in locale-dependent
+ * encoding, suitable to be given to CRT and/or GNUnet
+ */
+char *
+GNUNET_GTK_filechooser_get_filename_loc (GtkFileChooser *fc)
+{
+  char *filename_loc;
+  gchar *filename = gtk_file_chooser_get_filename (fc);
+  filename_loc = GNUNET_GTK_from_filename_to_loc (filename);
+  g_free (filename);
+  return filename_loc;
+}
+
+gchar *
+GNUNET_GTK_filechooser_get_filename_utf8 (GtkFileChooser *fc)
+{
+  char *filename_utf8;
+  gchar *filename = gtk_file_chooser_get_filename (fc);
+  filename_utf8 = GNUNET_GTK_from_filename_to_utf8 (filename);
+  g_free (filename);
+  return filename_utf8;
+}
+
 /* end of nls.c */
-- 
1.7.4
0001-Temporary-fix-for-charset-conversion.patch (33,178 bytes)   
0001-New-charset-conversion-functions.patch (4,327 bytes)    
From 2b3d150620970b011372749df22c03651c0e8208 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=A0=D1=83=D1=81=D0=BB=D0=B0=D0=BD=20=D0=98=D0=B6=D0=B1=D1?=
 =?UTF-8?q?=83=D0=BB=D0=B0=D1=82=D0=BE=D0=B2?= <lrn1986@gmail.com>
Date: Sat, 24 Dec 2011 23:09:04 +0400
Subject: [PATCH] New charset conversion functions

---
 src/include/gnunet_strings_lib.h |   23 ++++++++++++++++++++
 src/util/strings.c               |   43 +++++++++++++++++++++++++++++++------
 2 files changed, 59 insertions(+), 7 deletions(-)

diff --git a/src/include/gnunet_strings_lib.h b/src/include/gnunet_strings_lib.h
index 4efecc8..edeccaf 100644
--- a/src/include/gnunet_strings_lib.h
+++ b/src/include/gnunet_strings_lib.h
@@ -87,6 +87,18 @@ GNUNET_STRINGS_byte_size_fancy (unsigned long long size);
 
 /**
  * Convert the len characters long character sequence
+ * given in input that is in the given input charset
+ * to a string in given output charset.
+ * @return the converted string (0-terminated),
+ *  if conversion fails, a copy of the orignal
+ *  string is returned.
+ */
+char *
+GNUNET_STRINGS_conv (const char *input, size_t len,
+    const char *input_charset, const char *output_charset);
+
+/**
+ * Convert the len characters long character sequence
  * given in input that is in the given charset
  * to UTF-8.
  *
@@ -98,6 +110,17 @@ GNUNET_STRINGS_byte_size_fancy (unsigned long long size);
 char *
 GNUNET_STRINGS_to_utf8 (const char *input, size_t len, const char *charset);
 
+/**
+ * Convert the len bytes-long UTF-8 string
+ * given in input to the given charset.
+
+ * @return the converted string (0-terminated),
+ *  if conversion fails, a copy of the orignal
+ *  string is returned.
+ */
+char *
+GNUNET_STRINGS_from_utf8 (const char *input, size_t len, const char *charset);
+
 
 /**
  * Complete filename (a la shell) from abbrevition.
diff --git a/src/util/strings.c b/src/util/strings.c
index 2b5538b..bd0a8eb 100644
--- a/src/util/strings.c
+++ b/src/util/strings.c
@@ -327,17 +327,16 @@ GNUNET_STRINGS_fancy_time_to_relative (const char *fancy_size,
   return GNUNET_OK;
 }
 
-
 /**
  * Convert the len characters long character sequence
- * given in input that is in the given charset
- * to UTF-8.
+ * given in input that is in the given input charset
+ * to a string in given output charset.
  * @return the converted string (0-terminated),
  *  if conversion fails, a copy of the orignal
  *  string is returned.
  */
 char *
-GNUNET_STRINGS_to_utf8 (const char *input, size_t len, const char *charset)
+GNUNET_STRINGS_conv (const char *input, size_t len, const char *input_charset, const char *output_charset)
 {
   char *ret;
 
@@ -348,12 +347,12 @@ GNUNET_STRINGS_to_utf8 (const char *input, size_t len, const char *charset)
   char *itmp;
   iconv_t cd;
 
-  cd = iconv_open ("UTF-8", charset);
+  cd = iconv_open (output_charset, input_charset);
   if (cd == (iconv_t) - 1)
   {
     LOG_STRERROR (GNUNET_ERROR_TYPE_WARNING, "iconv_open");
-    LOG (GNUNET_ERROR_TYPE_WARNING, _("Character set requested was `%s'\n"),
-         charset);
+    LOG (GNUNET_ERROR_TYPE_WARNING, _("Character sets requested were `%s'->`%s'\n"),
+         input_charset, output_charset);
     ret = GNUNET_malloc (len + 1);
     memcpy (ret, input, len);
     ret[len] = '\0';
@@ -396,6 +395,36 @@ GNUNET_STRINGS_to_utf8 (const char *input, size_t len, const char *charset)
 
 
 /**
+ * Convert the len characters long character sequence
+ * given in input that is in the given charset
+ * to UTF-8.
+ * @return the converted string (0-terminated),
+ *  if conversion fails, a copy of the orignal
+ *  string is returned.
+ */
+char *
+GNUNET_STRINGS_to_utf8 (const char *input, size_t len, const char *charset)
+{
+  return GNUNET_STRINGS_conv (input, len, charset, "UTF-8");
+}
+
+/**
+ * Convert the len bytes-long UTF-8 string
+ * given in input to the given charset.
+
+ * @return the converted string (0-terminated),
+ *  if conversion fails, a copy of the orignal
+ *  string is returned.
+ */
+char *
+GNUNET_STRINGS_from_utf8 (const char *input, size_t len, const char *charset)
+{
+  return GNUNET_STRINGS_conv (input, len, "UTF-8", charset);
+}
+
+
+
+/**
  * Complete filename (a la shell) from abbrevition.
  * @param fil the name of the file, may contain ~/ or
  *        be relative to the current directory
-- 
1.7.4
0001-New-charset-conversion-functions.patch (4,327 bytes)   

Activities

LRN

2011-12-24 15:05

developer   ~0005190

Uploaded a big patch that, among other things, fixes charset conversions. It's still ugly, but at least now i am able to specify non-English filenames when downloading and publishing. It might require the patch from 0002032 to be applied first.

Christian Grothoff

2011-12-24 17:07

manager   ~0005193

Just to clarify: we want to use utf-8 internally everywhere. File-systems on modern GNU systems are already in utf-8, which means conversions there are a bit hard to test for me. But the correct thing to do would be to convert to/from Utf8 as soon as possible when interacting with a non-utf8 compliant API, and NOT at the gnunet--gnunet-gtk boundary.

Christian Grothoff

2011-12-24 17:19

manager   ~0005194

The patch as is massively violates our API naming conventions (no GNUNET_[fs_]GTK_ prefix in symbols exported). The same has applied to many of our signal handlers in the recent past. Could you please try to make sure to give the appropriate prefix to non-static symbols?

I also don't see why you use the Gtk string conversion functions instead of GNUNET_STRINGS_to_utf8. Overall, we clearly need to develop a clearer policy here, see for example:

- desc = GNUNET_strdup (_("no description supplied"));
+ desc = g_strdup (_("no description supplied"));

These two lines are NOT equivalent. GNUNET_strdup checks for out-of-memory and does NOT allow "NULL" as the first argument; both conditions result in an abort() and do not require further error checking on the client side.

g_strdup's behavior for out-of-memory is undocumented (I believe it would return NULL) and it does allow "NULL" as the argument to be given (in which case it also returns NULL). Neither behavior matches "strdup" from libc.

For GNUnet, we intentionally wrapped many libc APIs in "stronger" versions with harder error-checking so that the users would not have to always be so careful with error handling (especially for strdup/malloc-like situations). Using the *weaker-than-libc* functions from glib (where the design decision is clearly to be more tolerant in the case of errors and continue with the "hope" that things will work out) is IMO a bad idea for GNUnet: I don't see you add error checking (where do you ever check if the allocation succeeded in your patches?), and the GUI is just as critical for security as the rest of the system.

So in my opinion, where possible/applicable, we should use the GNUnet APIs instead of the glib APIs.

LRN

2011-12-24 17:24

developer   ~0005196

1) I'm all for earliest UTF-8 handling, but it requires considerable amount of hacking all over GNUnet. The boundary between GTK and GNUnet is much smaller, and thus easier to patch.

2) I can fix the names, no problem.

3) I'm using Glib functions instead of GNUnet ones in some places for consistency: the pointer must be freed EITHER by g_free(), OR by GNUNET_strdup(). Thus i must user either Glib functions to allocate it, or GNUnet ones. I chose Glib, because charset conversion functions also belong to Glib, and allocate with GLib calls, so it's not like i can use GNUnet for this.

LRN

2011-12-24 20:10

developer   ~0005197

Uploaded new version of the patch. It now uses gnunet charset conversion routines. Requires a patch to be applied to GNUnet itself (also attached).

Christian Grothoff

2011-12-24 22:45

manager   ~0005198

Patches applied in SVN 18801/18802. While this may not fix every single place where something might be wrong, I think the "big" issue is likely fixed by this, so I'm resolving this for now. As we discover other locations where the conversion is not done right, please open new bugs (and possibly refer to this one for discussion).

Issue History

Date Modified Username Field Change
2011-12-23 22:16 LRN New Issue
2011-12-23 22:16 LRN Status new => assigned
2011-12-23 22:16 LRN Assigned To => NDurner
2011-12-24 15:04 LRN File Added: 0001-Numerous-fixes.patch
2011-12-24 15:05 LRN Note Added: 0005190
2011-12-24 17:07 Christian Grothoff Note Added: 0005193
2011-12-24 17:19 Christian Grothoff Note Added: 0005194
2011-12-24 17:20 Christian Grothoff Summary No lowlevel UTF-8 support in GNUnet => [patch] No lowlevel UTF-8 support in GNUnet
2011-12-24 17:24 LRN Note Added: 0005196
2011-12-24 20:08 LRN File Added: 0001-Temporary-fix-for-charset-conversion.patch
2011-12-24 20:08 LRN File Deleted: 0001-Numerous-fixes.patch
2011-12-24 20:09 LRN File Added: 0001-New-charset-conversion-functions.patch
2011-12-24 20:10 LRN Note Added: 0005197
2011-12-24 22:45 Christian Grothoff Note Added: 0005198
2011-12-24 22:45 Christian Grothoff Status assigned => resolved
2011-12-24 22:45 Christian Grothoff Fixed in Version => 0.9.1
2011-12-24 22:45 Christian Grothoff Resolution open => fixed
2011-12-24 22:45 Christian Grothoff Assigned To NDurner => LRN
2011-12-25 17:49 Christian Grothoff Product Version => 0.9.0
2011-12-25 17:49 Christian Grothoff Target Version => 0.9.1
2011-12-26 22:28 Christian Grothoff Status resolved => closed
2024-01-12 14:28 schanzen Category Win32 port => Win32 port (deprecated)
2024-05-03 13:51 Christian Grothoff Category Win32 port (deprecated) => obsolete