View Issue Details

IDProjectCategoryView StatusLast Update
0006776Talerexchangepublic2021-03-01 14:11
ReporterChristian Grothoff Assigned ToChristian Grothoff  
PriorityurgentSeveritymajorReproducibilityrandom
Status resolvedResolutionfixed 
Platformi7OSDebian GNU/LinuxOS Versionsqueeze
Product Versiongit (master) 
Target Version0.9Fixed in Version0.9 
Summary0006776: taler-exchange-secmod-eddsa deadlocks on termination
DescriptionWe see this in production. Basically, all threads hang in a printf() log statement, except main, which hangs in a pthread_join().

[Switching to thread 1 (Thread 0x7f4b35e13300 (LWP 1376407))]
#0 __pthread_clockjoin_ex (threadid=139960983385856, thread_return=0x0,
clockid=<optimized out>,
     abstime=<optimized out>, block=<optimized out>) at
pthread_join_common.c:145
145 pthread_join_common.c: No such file or directory.
(gdb) ba
#0 __pthread_clockjoin_ex (threadid=139960983385856, thread_return=0x0,
clockid=<optimized out>,
     abstime=<optimized out>, block=<optimized out>) at
pthread_join_common.c:145
#1 0x000055d1cb301484 in do_shutdown (cls=0x0) at
taler-exchange-secmod-eddsa.c:1438
#2 0x00007f4b369af9ab in GNUNET_SCHEDULER_do_work
(sh=sh@entry=0x55d1cbcca760) at scheduler.c:2144
#3 0x00007f4b369b09e9 in select_loop (context=0x7ffd5b99a8d0,
sh=0x55d1cbcca760) at scheduler.c:2441
#4 GNUNET_SCHEDULER_run (task=task@entry=0x7f4b369a5fc0 <program_main>,
task_cls=task_cls@entry=0x7ffd5b99a990)
     at scheduler.c:738
#5 0x00007f4b369a66a7 in GNUNET_PROGRAM_run2 (argc=<optimized out>,
argv=0x7ffd5b99add8,
     binaryName=<optimized out>, binaryHelp=<optimized out>,
options=<optimized out>, task=<optimized out>,
     task_cls=<optimized out>, run_without_scheduler=<optimized out>) at
program.c:344
#6 0x00007f4b369a6c7f in GNUNET_PROGRAM_run (argc=<optimized out>,
argv=<optimized out>,
     binaryName=<optimized out>, binaryHelp=<optimized out>,
options=<optimized out>, task=<optimized out>,
     task_cls=0x0) at program.c:371
#7 0x000055d1cb3020d2 in main (argc=3, argv=0x7ffd5b99add8) at
taler-exchange-secmod-eddsa.c:1692
Detaching from program:
/home/taler-test/local/bin/taler-exchange-secmod-eddsa, process 1376407


(gdb) info threads
   Id Target Id Frame
   1 Thread 0x7f4b35e13300 (LWP 1376407) "taler-exchange-"
__pthread_clockjoin_ex (threadid=139960983385856,
     thread_return=0x0, clockid=<optimized out>, abstime=<optimized out>,
block=<optimized out>)
     at pthread_join_common.c:145
   2 Thread 0x7f4b34b27700 (LWP 1376427) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
   3 Thread 0x7f4b2ffff700 (LWP 1376428) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
   4 Thread 0x7f4b2dffb700 (LWP 1376432) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
   5 Thread 0x7f4b2d7fa700 (LWP 1376433) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
   6 Thread 0x7f4b12ffd700 (LWP 1376437) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
   7 Thread 0x7f4b127fc700 (LWP 1376438) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35
* 8 Thread 0x7f4b11ffb700 (LWP 1376439) "taler-exchange-"
__lll_lock_wait_private (futex=0x55d1cbcc3200)
     at ./lowlevellock.c:35

[Switching to thread 7 (Thread 0x7f4b127fc700 (LWP 1376438))]
#0 __lll_lock_wait_private (futex=0x55d1cbcc3200) at ./lowlevellock.c:35
35 in ./lowlevellock.c
(gdb) ba
#0 __lll_lock_wait_private (futex=0x55d1cbcc3200) at ./lowlevellock.c:35
#1 0x00007f4b367c6aa6 in __vfprintf_internal (s=0x55d1cbcc3120,
format=0x7f4b369bfdf7 "%s %s %s %s",
     ap=ap@entry=0x7f4b127fafe0, mode_flags=mode_flags@entry=0) at
vfprintf-internal.c:1370
#2 0x00007f4b367b1cc6 in __fprintf (stream=<optimized out>,
format=format@entry=0x7f4b369bfdf7 "%s %s %s %s")
     at fprintf.c:32
#3 0x00007f4b369728c8 in output_message
(kind=kind@entry=GNUNET_ERROR_TYPE_DEBUG,
     comp=comp@entry=0x55d1cbcc9150 "taler-exchange-secmod-eddsa-1376407",
     datestr=datestr@entry=0x7f4b127fb170 "Feb 17 08:00:40-694075",
     msg=msg@entry=0x7f4b127fb100 "Crypto worker exiting\n") at
common_logging.c:854
#4 0x00007f4b36972e4c in mylog (kind=GNUNET_ERROR_TYPE_DEBUG,
     comp=0x55d1cbcc9150 "taler-exchange-secmod-eddsa-1376407",
message=0x55d1cb3030eb "Crypto worker exiting\n",
     va=<optimized out>) at common_logging.c:1045
#5 0x00007f4b369730e4 in GNUNET_log_nocheck (kind=<optimized out>,
message=<optimized out>)
     at common_logging.c:1063
#6 0x000055d1cb2fd106 in sign_worker (cls=0x0) at
taler-exchange-secmod-eddsa.c:402
#7 0x00007f4b36928ea7 in start_thread (arg=<optimized out>) at
pthread_create.c:477
#8 0x00007f4b36858def in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95

TagsNo tags attached.

Activities

Christian Grothoff

2021-03-01 14:11

manager   ~0017579

Removed the deadlocking log statements for now. This really smells like a libc issue, at least I again could find nothing else to blame.

Issue History

Date Modified Username Field Change
2021-03-01 13:28 Christian Grothoff New Issue
2021-03-01 13:28 Christian Grothoff Status new => assigned
2021-03-01 13:28 Christian Grothoff Assigned To => Christian Grothoff
2021-03-01 14:11 Christian Grothoff Status assigned => resolved
2021-03-01 14:11 Christian Grothoff Resolution open => fixed
2021-03-01 14:11 Christian Grothoff Fixed in Version => 0.9
2021-03-01 14:11 Christian Grothoff Note Added: 0017579
2021-03-01 14:11 Christian Grothoff Product Version => git (master)
2021-03-01 14:11 Christian Grothoff Target Version => 0.9