View Issue Details

IDProjectCategoryView StatusLast Update
0001373libextractorlibextractor main librarypublic2012-09-09 02:36
ReporterholinAssigned Toholin 
Status closedResolutionfixed 
Product Versioncurrent SVN 
Target Version1.0.0Fixed in Version1.0.0 
Summary0001373: module handling in LE is not thread-safe
DescriptionRecently, I added a multithread test case to LE. Running it shows crashes in seemingly random places. This was observed on OS X. I tracked the problem to module loading/unloading, and reading the libtool docs shows the reason: libtool (libltdl) module loading is not thread-safe, and LE uses libtool functions without protection. I see no way around this other than adding locking around libtool functions (or documentating that all module handling is not thread-safe.) I suppose this applies to GNUnet as well.
TagsNo tags attached.



2008-06-22 07:32

developer   ~0003520

link to relevant docs:

Christian Grothoff

2008-06-22 09:16

manager   ~0003521

Well, in GNUnet we have already the strong assumption that all loading/unloading is done in single-threaded mode during startup/shutdown. Everything else will break tons of stuff.

For LE -- well, the assumption is that obviously the user does not load/unload on the same handle while also calling extract on that handle (that would be invalid). If the problem is that two threads cannot call, say, load-default-extractors at the same time, well, then we have to add some locking.


2008-06-22 09:27

developer   ~0003522

Yes. The problem is that calling any of the functions that call libltdl functions (eg. lt_dlopenex) unprotected is not thread-safe. For instance, EXTRACTOR_loadDefaultLibraries is not thread-safe.

Of course, sharing a handle between threads would be asking for trouble anyway, and I'm not talking about any weird programming practices here. Unless..unless an EXTRACTOR_KeywordList would also be invalid after freeing the EXTRACTOR_ExtractorList that extracted it, because I'm freeing the extractors before the keywordlist in my testcase.

Christian Grothoff

2008-06-22 10:00

manager   ~0003525

Sharing a handle between threads is *legal*, as long as all of the threads use it "read only" (reading == extracting, writing == manipulating plugin set).

And no, the keyword list is independent of the extractor list (and so freeing them in any order is fine).

So yes, overall this sounds like we may want to add locking around the ltdl functions (and only around those functions).


2008-06-22 12:49

developer   ~0003530

Fixed in SVN 7306. Tested on Mac OS X/ppc and Ubuntu 8.04/x86.


2008-06-22 12:50

developer   ~0003531

Documentation needs an update also.

Christian Grothoff

2008-06-22 22:49

manager   ~0003533

Are you going to do the update?


2008-06-23 04:26

developer   ~0003536

Yeah, I can do it.


2008-06-23 10:09

developer   ~0003537

Took a stab at updating the docs -- the texinfo doc was the only place where reentrancy was mentioned, so, I updated only that. SVN 7311.


2008-06-23 10:10

developer   ~0003538

Should be fixed in SVN 7311.

Issue History

Date Modified Username Field Change
2008-06-22 04:19 holin New Issue
2008-06-22 07:32 holin Note Added: 0003520
2008-06-22 09:16 Christian Grothoff Note Added: 0003521
2008-06-22 09:27 holin Note Added: 0003522
2008-06-22 10:00 Christian Grothoff Note Added: 0003525
2008-06-22 10:54 holin Status new => assigned
2008-06-22 10:54 holin Assigned To => holin
2008-06-22 12:49 holin Note Added: 0003530
2008-06-22 12:50 holin Note Added: 0003531
2008-06-22 22:49 Christian Grothoff Note Added: 0003533
2008-06-23 04:26 holin Note Added: 0003536
2008-06-23 10:09 holin Note Added: 0003537
2008-06-23 10:10 holin Status assigned => resolved
2008-06-23 10:10 holin Fixed in Version => current SVN
2008-06-23 10:10 holin Resolution open => fixed
2008-06-23 10:10 holin Note Added: 0003538
2008-07-13 19:40 Christian Grothoff Status resolved => closed
2012-09-09 02:36 Christian Grothoff Fixed in Version current SVN => 1.0.0
2012-09-09 02:36 Christian Grothoff Target Version => 1.0.0