View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0002744 | GNUnet | DNS service | public | 2013-01-25 13:41 | 2013-02-05 21:42 |
| Reporter | Christian Grothoff | Assigned To | Christian Grothoff | ||
| Priority | high | Severity | block | Reproducibility | always |
| Status | closed | Resolution | fixed | ||
| Product Version | Git master | ||||
| Target Version | 0.9.5a | Fixed in Version | 0.9.5a | ||
| Summary | 0002744: gnunet-service-dns takes 100% cpu | ||||
| Description | On toxic, after running the peer (with Tor) for a while, gnunet-service-dns begins to take over the CPU. Attaching gdb gives: Reading symbols from /lib/i386-linux-gnu/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/i386-linux-gnu/libgpg-error.so.0 0xb7751424 in __kernel_vsyscall () (gdb) ba #0 0xb7751424 in __kernel_vsyscall () #1 0xb7604ec1 in sendto () at ../sysdeps/unix/sysv/linux/i386/socket.S:61 #2 0xb76f7401 in GNUNET_NETWORK_socket_sendto (desc=0x9bb9150, message=0x9bb6240, length=1178, dest_addr=0x8b62a6c, dest_len=16) at network.c:726 #3 0xb774156a in GNUNET_DNSSTUB_resolve (ctx=0x9ba8058, sa=0x8b62a6c, sa_len=16, request=0x9bb6240, request_len=1178, rc=0x804ab64 <process_dns_result>, rc_cls=0x0) at dnsstub.c:289 #4 0x0804a31d in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:649 #5 0x0804a1f0 in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:633 #6 0x0804a153 in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:622 #7 0x0804bdd6 in process_helper_messages (cls=0x0, client=0x0, message=0x9bbac90) at gnunet-service-dns.c:1143 #8 0xb770acf6 in GNUNET_SERVER_mst_receive (mst=0x9bad558, client_identity=0x0, buf=0xbfcd0a38 "_\352\004\b&", size=0, purge=0, one_shot=0) at server_mst.c:224 #9 0xb76f4e97 in helper_read (cls=0x9bad4d8, tc=0xbfcdf7c0) at helper.c:302 #10 0xb7704fca in run_ready (rs=0x9bb5040, ws=0x9bb50c8) at scheduler.c:597 #11 0xb770587f in GNUNET_SCHEDULER_run (task=0xb77116dd <service_task>, task_cls=0xbfcdfa04) at scheduler.c:786 #12 0xb771329b in GNUNET_SERVICE_run (argc=3, argv=0xbfcdfbe4, service_name=0x804cc72 "dns", options=GNUNET_SERVICE_OPTION_NONE, task=0x804c0ea <run>, task_cls=0x0) at service.c:1815 #13 0x0804c8b8 in main (argc=3, argv=0xbfcdfbe4) at gnunet-service-dns.c:1420 (gdb) up #1 0xb7604ec1 in sendto () at ../sysdeps/unix/sysv/linux/i386/socket.S:61 #2 0xb76f7401 in GNUNET_NETWORK_socket_sendto (desc=0x9bb9150, message=0x9bb6240, length=1178, dest_addr=0x8b62a6c, dest_len=16) at network.c:726 #3 0xb774156a in GNUNET_DNSSTUB_resolve (ctx=0x9ba8058, sa=0x8b62a6c, sa_len=16, request=0x9bb6240, request_len=1178, rc=0x804ab64 <process_dns_result>, rc_cls=0x0) at dnsstub.c:289 #4 0x0804a31d in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:649 #5 0x0804a1f0 in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:633 (gdb) print *rr $1 = {client_wait_list = 0x0, payload = 0x9bb6240 "\004\232", rs = 0x9ba8be0, src_addr = {ss_family = 2, __ss_align = 16907945, __ss_padding = '\000' <repeats 119 times>}, dst_addr = { ss_family = 2, __ss_align = 4245421737, __ss_padding = '\000' <repeats 119 times>}, request_id = 287161287350788, payload_length = 1178, client_wait_list_length = 0, phase = RP_INTERNET_DNS} (gdb) up #6 0x0804a153 in next_phase (rr=0x8b629e0) at gnunet-service-dns.c:622 (gdb) up #7 0x0804bdd6 in process_helper_messages (cls=0x0, client=0x0, message=0x9bbac90) at gnunet-service-dns.c:1143 (gdb) print *message $3 = {size = 48644, type = 54784} (gdb) (gdb) print *ipv4 No symbol "ipv4" in current context. (gdb) print *ip4 $6 = {header_length = 5, version = 4, diff_serv = 0 '\000', total_length = 46596, identification = 0, flags = 0, fragmentation_offset = 8, ttl = 64 '@', protocol = 17 '\021', checksum = 15573, source_address = {s_addr = 16907945}, destination_address = {s_addr = 4245421737}} (gdb) print *udp $7 = {source_port = 19630, destination_port = 9736, len = 41476, crc = 46255} (gdb) print *dns $8 = {id = 39428, flags = 14336, qdcount = 0, ancount = 0, nscount = 10458, arcount = 49387} | ||||
| Tags | No tags attached. | ||||
|
|
exit-gnunet Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.0.0.0 P-t-P:10.0.0.0 Mask:255.0.0.0 inet6 addr: 2a00:4700:0:3:211:2fff:0:1/64 Scope:Global UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:105660 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 B) TX bytes:22249299 (21.2 MiB) gnunet-dns Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:169.254.1.1 P-t-P:169.254.1.1 Mask:255.255.0.0 inet6 addr: 2001:db8::1/126 Scope:Global UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:89325 errors:0 dropped:0 overruns:0 frame:0 TX packets:87024136 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:8656838 (8.2 MiB) TX bytes:1634781559 (1.5 GiB) |
|
|
Traffic on gnunet-dns link does in fact increase rapidly. |
|
|
Tor complains: Jan 25 11:17:32.000 [warn] eventdns: All nameservers have failed Jan 25 11:17:47.000 [notice] eventdns: Nameserver 8.8.4.4:53 is back up Jan 25 11:26:03.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:03.000 [notice] eventdns: Nameserver 8.8.8.8:53 is back up Jan 25 11:26:03.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:03.000 [notice] eventdns: Nameserver 8.8.8.8:53 is back up Jan 25 11:26:03.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:03.000 [notice] eventdns: Nameserver 8.8.8.8:53 is back up Jan 25 11:26:03.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:03.000 [notice] eventdns: Nameserver 8.8.8.8:53 is back up Jan 25 11:26:08.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:08.000 [notice] eventdns: Nameserver 8.8.4.4:53 is back up Jan 25 11:26:13.000 [warn] eventdns: All nameservers have failed Jan 25 11:26:22.000 [notice] eventdns: Nameserver 8.8.8.8:53 is back up Jan 25 13:25:28.000 [warn] eventdns: All nameservers have failed Jan 25 13:25:28.000 [notice] eventdns: Nameserver 8.8.4.4:53 is back up |
|
|
(gdb) up #1 0xb7604ec1 in sendto () at ../sysdeps/unix/sysv/linux/i386/socket.S:61 #2 0xb76f7401 in GNUNET_NETWORK_socket_sendto (desc=0x9bbac78, message=0x9c3c770, length=1178, dest_addr=0x8b62a6c, dest_len=16) at network.c:726 (gdb) print *dest_addr $1 = {sa_family = 2, sa_data = "\b&\251\376\fn\000\000\000\000\000\000\000"} (gdb) print (struct sockaddr_in*) dest_addr $2 = (struct sockaddr_in *) 0x8b62a6c (gdb) print *(struct sockaddr_in*) dest_addr $3 = {sin_family = 2, sin_port = 9736, sin_addr = {s_addr = 1846345385}, sin_zero = "\000\000\000\000\000\000\000"} Note: that's port 2086 on some non-DNS resolver's IP address. Why do we send traffic there!? |
|
|
# iptables-save # Generated by iptables-save v1.4.16.3 on Fri Jan 25 13:38:55 2013 *mangle :PREROUTING ACCEPT [34365315:23326521846] :INPUT ACCEPT [34365315:23326521846] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [4422517979:5311512830604] :POSTROUTING ACCEPT [4422734341:5311535365642] -A OUTPUT -p udp -m owner --gid-owner 1001 -m udp --dport 53 -j ACCEPT -A OUTPUT -p udp -m udp --dport 53 -j MARK --set-xmark 0x8260035/0xffffffff COMMIT # Completed on Fri Jan 25 13:38:55 2013 # Generated by iptables-save v1.4.16.3 on Fri Jan 25 13:38:55 2013 *filter :INPUT ACCEPT [454898212:260727078140] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [13355851518:16010191948774] COMMIT # Completed on Fri Jan 25 13:38:55 2013 # Generated by iptables-save v1.4.16.3 on Fri Jan 25 13:38:55 2013 *nat :PREROUTING ACCEPT [3336919:209523422] :INPUT ACCEPT [3344288:209949986] :OUTPUT ACCEPT [1812431:280610271] :POSTROUTING ACCEPT [263356:67041547] -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 1080 -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 4433 -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -o eth0 -j MASQUERADE COMMIT # Completed on Fri Jan 25 13:38:55 2013 |
|
|
UID info for gnunet-service-dns: (gdb) print getgid () $1 = 1001 (gdb) print getuid () $3 = 0 (gdb) print getegid() $1 = 1001 |
|
|
root@toxic:/var/log# ip rule 0: from all lookup local 32765: from all fwmark 0x8260035 lookup 53 32766: from all lookup main 32767: from all lookup default root@toxic:/var/log# ip route default via 188.95.234.2 dev eth0 10.0.0.0/8 dev exit-gnunet proto kernel scope link src 10.0.0.0 169.254.0.0/16 dev gnunet-dns proto kernel scope link src 169.254.1.1 188.95.234.0/23 dev eth0 proto kernel scope link src 188.95.234.4 root@toxic:/var/log# ip route list table 53 default dev gnunet-dns scope link |
|
|
Problem disappears after restart -- for a while, but then seems to eventually come back. |
|
|
Added a filter to drop packets to ports other than 53, seems to help so far. |
|
|
Couldn't reproduce after the port fix. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2013-01-25 13:41 | Christian Grothoff | New Issue | |
| 2013-01-25 13:42 | Christian Grothoff | Note Added: 0006797 | |
| 2013-01-25 13:43 | Christian Grothoff | Note Added: 0006798 | |
| 2013-01-25 13:44 | Christian Grothoff | Note Added: 0006799 | |
| 2013-01-25 13:46 | Christian Grothoff | Note Added: 0006800 | |
| 2013-01-25 13:47 | Christian Grothoff | Note Added: 0006801 | |
| 2013-01-25 13:49 | Christian Grothoff | Note Added: 0006802 | |
| 2013-01-25 13:49 | Christian Grothoff | Note Edited: 0006802 | |
| 2013-01-25 13:54 | Christian Grothoff | Note Added: 0006803 | |
| 2013-01-25 13:55 | Christian Grothoff | Note Edited: 0006803 | |
| 2013-01-25 13:56 | Christian Grothoff | Note Edited: 0006803 | |
| 2013-01-25 13:58 | Christian Grothoff | Note Edited: 0006800 | |
| 2013-01-25 14:15 | Christian Grothoff | Note Edited: 0006803 | |
| 2013-01-25 14:47 | Christian Grothoff | Note Added: 0006804 | |
| 2013-01-25 17:13 | Christian Grothoff | Note Added: 0006807 | |
| 2013-01-29 21:34 | Christian Grothoff | Note Added: 0006809 | |
| 2013-01-29 21:34 | Christian Grothoff | Status | new => resolved |
| 2013-01-29 21:34 | Christian Grothoff | Fixed in Version | => Git master |
| 2013-01-29 21:34 | Christian Grothoff | Resolution | open => fixed |
| 2013-01-29 21:34 | Christian Grothoff | Assigned To | => Christian Grothoff |
| 2013-01-29 21:34 | Christian Grothoff | Fixed in Version | Git master => 0.10.0 |
| 2013-01-29 21:34 | Christian Grothoff | Target Version | => 0.10.0 |
| 2013-02-05 21:40 | Christian Grothoff | Fixed in Version | 0.10.0 => 0.9.5a |
| 2013-02-05 21:40 | Christian Grothoff | Target Version | 0.10.0 => 0.9.5a |
| 2013-02-05 21:42 | Christian Grothoff | Status | resolved => closed |
