View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0002549GNUnettransport servicepublic2012-09-17 15:522024-05-03 13:58
Reporteraitap Assigned ToMatthias Wachs  
PriorityurgentSeveritymajorReproducibilitysometimes
Status closedResolutionfixed 
PlatformLinux/x86OSDebianOS Versionwheezy/sid
Product Version0.9.3 
Target Version0.9.4 
Summary0002549: UDP transport floods bandwidth sometimes
DescriptionApproximately once on a few hours gnunet-service-transport starts consuming 100% of one CPU core and flooding the bandwidth with outgoing UDP packets to another GNUnet node.

Also, other nodes can be observed flooding my node, and sometimes they do it simultaneously in two directions (both ingoing and outgoung).
Steps To ReproduceRun GNUnet node for some time with UDP transport enabled. No special actions requred (heisenbug).
Additional InformationOutput of `gnunet-statistics` is attached, as suggested at #gnunet.
TagsNo tags attached.
Attached Files
gnunet-statistics.1347889453 (14,219 bytes)

Relationships

related to 0002589 closedMatthias Wachs Transport plugins have to report overhead used to send message 

Activities

Matthias Wachs

2012-09-27 11:13

reporter   ~0006375

UDP passes quota compliance tests, BUT

quota is enforced on payload clients want to send, not on amount of data UDP is really sending (ACKs, retransmits, ... )

This can be clearly seen if you compare the to statistics values:
transport : "# bytes transmitted via UDP" (bytes succesfully transmitted with GNUNET_NETWORK_socket_sendto)

transport : "# bytes payload asked to transmit via UDP" (bytes given to plugin with plugin_send)

Matthias Wachs

2012-09-27 11:27

reporter   ~0006376

Last edited: 2012-09-27 11:28

 Real outbound rate is not limited
MAX_BPS and bandwidth tracker is not used in plugin

Matthias Wachs

2012-09-27 11:41

reporter   ~0006377

Plan:
For each message:
  last_diff = how many bytes are sent on the wire - message size
Return on next udp_plugin_send: msg_size + last_diff

Christian Grothoff

2012-10-06 20:42

manager   ~0006409

Also interesting: if I run (behind my home NAT with hole punched) TCP+UDP, I get ZERO connections (gnunet-transport -i and also for gnunet-core). Once I *disable* UDP, I instantly have a dozen connections. So UDP is _really_ problematic right now.

Matthias Wachs

2012-10-17 18:08

reporter   ~0006446

OK, I fixed some strange WTF bugs...

Please test with revision >= 24361

Matthias Wachs

2012-10-17 18:09

reporter   ~0006447

If the bug appears again please post the UDP related statistic values
Especially:


    transport # bytes overhead transmitted via UDP:
    transport # bytes payload transmitted via UDP:
    transport # bytes transmitted via UDP:
    transport # messages ACKs transmitted via UDP:

Christian Grothoff

2012-10-17 22:15

manager   ~0006448

Ok, it just went crazy again on my system. I have (de-facto) 64 kb/s upstream, GNUnet is configured to do 50 kb/s. iptraf reported 500+ kb/s UDP traffic, all other connections "failed" hard. I then took two gnunet-statistics snapshots 60s apart (not sure if the full 50 kb/s were going on the entire time). Here is the data:

grothoff@spec:~$ date ; gnunet-statistics -s transport ; sleep 60 ; gnunet-statistics -s transport
Wed Oct 17 22:12:26 CEST 2012
    transport # msgs currently in UDP buffers: 0
    transport # bytes currently in UDP buffers: 0
    transport # fragments wrap arounds: 215979
    transport # fragments retransmitted: 577691
    transport # fragments transmitted: 579585
    transport # bytes transmitted via UDP: 591537555
    transport # bytes transmitted via TCP: 16339776
    transport # bytes currently in TCP buffers: 492
    transport # messages transmitted to other peers: 19594
    transport # bytes in message queue for other peers: 23080
    transport # bytes payload received: 28507421
    transport # bytes total received: 31104388
    transport # bytes received via TCP: 27125120
    transport # bytes received via UDP: 66597988
    transport # transmission failures for messages to other peers: 47
    transport # bytes discarded by TCP (disconnect): 258884
    transport # TCP sessions active: 42
    transport # network-level TCP disconnect events: 1374
    transport # peers connected: 23
    transport # DISCONNECT messages sent: 48
    transport # REQUEST CONNECT messages received: 1604
    transport # keepalives sent: 4913
    transport # PONGs dropped, no matching pending validation: 9
    transport # PONG messages received: 3042
    transport # KEEPALIVE_RESPONSE messages discarded (not expected): 59
    transport # PING messages received: 2989
    transport # other peer asked to disconnect from us: 7
    transport # address records discarded: 1438
    transport # UDP sessions active: 25
    transport # TCP WELCOME messages received: 482
    transport # IPv4 broadcast HELLO beacons received via udp: 920
    transport # PING without HELLO messages sent: 4114
    transport # address revalidations started: 4610
    transport # bytes payload transmitted via UDP: 3687755
    transport # bytes overhead transmitted via UDP: 278428784
    transport # unfragmented messages transmit with success via UDP: 54259
    transport # unfragmented messages asked to transmit via UDP: 3597
    transport # bytes payload asked to transmit via UDP: 3725487
    transport # transport-service disconnect requests for TCP: 57
    transport # ms throttling suggested: 29000125
    transport # messages ACKs transmitted via UDP: 50662
    transport # acknowledgements sent for fragment: 50662
    transport # duplicate fragments received: 53320
    transport # fragments received: 54984
    transport # messages defragmented: 487
    transport # fragment acknowledgements received: 797
    transport # total size of fragmented messages: 2374783
    transport # messages fragmented: 337
    transport # fragmented messages asked to transmit via UDP: 337
    transport # fragmentation transmissions completed: 334
    transport # PING message for different peer received: 10
    transport # bytes payload discarded due to not connected peer : 1300464
    transport # messages timed out while in transport queue: 40
    transport # unexpected SESSION ACK messages: 201
    transport # refreshed my HELLO: 45
    transport # KEEPALIVE messages discarded (no session): 12
    transport # bytes discarded by TCP (timeout): 394
    transport # unexpected CONNECT_ACK messages (disconnecting): 2
    transport # disconnect messages ignored (timestamp): 1
    transport # PONGs unicast via reliable transport: 201
    transport # peak heap size: 2019328
    transport # peak resident set size: 4100096


    transport # bytes transmitted via TCP: 16379914
    transport # bytes currently in TCP buffers: 2566
    transport # messages transmitted to other peers: 19726
    transport # bytes in message queue for other peers: 18488
    transport # UDP sessions active: 26
    transport # PING messages received: 3008
    transport # bytes received via TCP: 27203700
    transport # bytes total received: 31204069
    transport # TCP sessions active: 49
    transport # TCP WELCOME messages received: 511
    transport # bytes payload transmitted via UDP: 3721149
    transport # bytes overhead transmitted via UDP: 612734612
    transport # unfragmented messages transmit with success via UDP: 54336
    transport # bytes transmitted via UDP: 617099801
    transport # msgs currently in UDP buffers: 0
    transport # bytes currently in UDP buffers: 0
    transport # unfragmented messages asked to transmit via UDP: 3674
    transport # bytes payload asked to transmit via UDP: 3757297
    transport # ms throttling suggested: 30249749
    transport # bytes payload received: 28581866
    transport # bytes received via UDP: 66738081
    transport # REQUEST CONNECT messages received: 1685
    transport # keepalives sent: 4936
    transport # PONG messages received: 3057
    transport # PING without HELLO messages sent: 4137
    transport # address revalidations started: 4636
    transport # peers connected: 24
    transport # KEEPALIVE_RESPONSE messages discarded (not expected): 63
    transport # transport-service disconnect requests for TCP: 72
    transport # DISCONNECT messages sent: 60
    transport # IPv4 broadcast HELLO beacons received via udp: 924
    transport # bytes discarded by TCP (disconnect): 261184
    transport # network-level TCP disconnect events: 1387
    transport # PONGs dropped, no matching pending validation: 10
    transport # other peer asked to disconnect from us: 16
    transport # fragmentation transmissions completed: 336
    transport # fragment acknowledgements received: 799
    transport # fragments retransmitted: 606156
    transport # fragments transmitted: 608063
    transport # fragments wrap arounds: 228916
    transport # total size of fragmented messages: 2392395
    transport # messages fragmented: 338
    transport # fragmented messages asked to transmit via UDP: 338
    transport # unexpected SESSION ACK messages: 202
    transport # transmission failures for messages to other peers: 49
    transport # address records discarded: 1441
    transport # messages ACKs transmitted via UDP: 50662
    transport # acknowledgements sent for fragment: 50662
    transport # duplicate fragments received: 53320
    transport # fragments received: 54984
    transport # messages defragmented: 487
    transport # PING message for different peer received: 10
    transport # bytes payload discarded due to not connected peer : 1300464
    transport # messages timed out while in transport queue: 40
    transport # refreshed my HELLO: 45
    transport # KEEPALIVE messages discarded (no session): 12
    transport # bytes discarded by TCP (timeout): 394
    transport # unexpected CONNECT_ACK messages (disconnecting): 2
    transport # disconnect messages ignored (timestamp): 1
    transport # PONGs unicast via reliable transport: 201
    transport # peak heap size: 2019328
    transport # peak resident set size: 4100096

Matthias Wachs

2012-10-18 14:58

reporter   ~0006451

After analysis:

UDP is quota compliant
...in respect to transport payload, but not with overhead!

Here some data:
Sent total:
    transport # UDP, total, bytes overhead, sent: 289703914
    transport # UDP, total, bytes payload, sent: 3481894
    transport # UDP, total, bytes, sent, success: 293343916


    transport # UDP, unfragmented msgs, bytes payload, sent, success: 1197995
    transport # UDP, unfragmented msgs, bytes overhead, sent, success: 276264

    transport # UDP, fragmented msgs, bytes payload, sent, success: 2290689
    transport # UDP, fragmented msgs, bytes overhead, sent, success: 285775566

    transport # UDP, fragmented msgs, fragments bytes, sent, failure: 1400
--> transport # UDP, fragmented msgs, fragments bytes, sent, success: 288046071
    transport # UDP, fragmented msgs, fragments, sent, failure: 1
--> transport # UDP, fragmented msgs, fragments, sent, success: 269240

Matthias Wachs

2012-10-18 15:58

reporter   ~0006452

Analysis for fragmented messges:

All fragmented messages were successfully sent
No fragmented messages is stuck in send queue (continous sending)

-> The overhead is generated from successful send operations!

Matthias Wachs

2012-10-18 16:57

reporter   ~0006453

Retransmit are killing us:

Oct 18 16:54:05-454003 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 9 , payload 2900 bytes, used on wire 9024 bytes, overhead: 3.111724, expected delay: 1000
Oct 18 16:54:06-216475 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 40 , payload 6256 bytes, used on wire 51104 bytes, overhead: 8.168797, expected delay: 102
Oct 18 16:54:11-026793 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 36 , payload 4533 bytes, used on wire 41877 bytes, overhead: 9.238253, expected delay: 77
Oct 18 16:54:12-405682 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 9 , payload 3431 bytes, used on wire 10617 bytes, overhead: 3.094433, expected delay: 792
Oct 18 16:54:14-630059 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 29 , payload 39245 bytes, used on wire 39665 bytes, overhead: 1.010702, expected delay: 161
Oct 18 16:54:19-444063 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 12 , payload 3728 bytes, used on wire 15344 bytes, overhead: 4.115880, expected delay: 597
Oct 18 16:54:23-836477 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 12 , payload 3128 bytes, used on wire 12944 bytes, overhead: 4.138107, expected delay: 448
Oct 18 16:54:28-863418 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 10 , payload 2136 bytes, used on wire 11160 bytes, overhead: 5.224719, expected delay: 354
Oct 18 16:54:31-941543 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 15 , payload 3072 bytes, used on wire 15900 bytes, overhead: 5.175781, expected delay: 301
Oct 18 16:54:34-036621 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 38 , payload 1752 bytes, used on wire 35112 bytes, overhead: 20.041096, expected delay: 10
Oct 18 16:54:35-016609 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 20 , payload 5136 bytes, used on wire 26280 bytes, overhead: 5.116822, expected delay: 226
Oct 18 16:54:40-819437 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 16 , payload 1496 bytes, used on wire 12736 bytes, overhead: 8.513369, expected delay: 179
Oct 18 16:54:43-082893 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 20 , payload 12700 bytes, used on wire 25784 bytes, overhead: 2.030236, expected delay: 122
Oct 18 16:54:45-584839 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 14 , payload 2152 bytes, used on wire 15736 bytes, overhead: 7.312268, expected delay: 145
Oct 18 16:54:53-125119 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 24 , payload 3896 bytes, used on wire 32032 bytes, overhead: 8.221766, expected delay: 121
Oct 18 16:54:54-452695 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 14 , payload 8785 bytes, used on wire 17882 bytes, overhead: 2.035515, expected delay: 91
Oct 18 16:55:02-254243 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 4 , payload 1680 bytes, used on wire 3552 bytes, overhead: 2.114286, expected delay: 69
Oct 18 16:55:04-653165 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 6 , payload 2376 bytes, used on wire 7416 bytes, overhead: 3.121212, expected delay: 59
Oct 18 16:55:05-033044 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 70 , payload 8473 bytes, used on wire 86290 bytes, overhead: 10.184114, expected delay: 60
Oct 18 16:55:07-089962 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 9 , payload 3600 bytes, used on wire 11124 bytes, overhead: 3.090000, expected delay: 44
Oct 18 16:55:07-463800 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 30 , payload 3260 bytes, used on wire 33680 bytes, overhead: 10.331288, expected delay: 45
Oct 18 16:55:10-366474 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 15 , payload 6456 bytes, used on wire 19764 bytes, overhead: 3.061338, expected delay: 34
Oct 18 16:55:10-913580 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 28 , payload 2088 bytes, used on wire 30576 bytes, overhead: 14.643678, expected delay: 36
Oct 18 16:55:11-596717 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 42 , payload 6959 bytes, used on wire 49721 bytes, overhead: 7.144848, expected delay: 98
Oct 18 16:55:12-600447 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 48 , payload 1972 bytes, used on wire 49632 bytes, overhead: 25.168358, expected delay: 8
Oct 18 16:55:13-898706 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 70 , payload 6381 bytes, used on wire 91182 bytes, overhead: 14.289610, expected delay: 28
Oct 18 16:55:16-065319 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 8 , payload 2344 bytes, used on wire 9760 bytes, overhead: 4.163823, expected delay: 26
Oct 18 16:55:16-251354 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 55 , payload 6565 bytes, used on wire 73667 bytes, overhead: 11.221173, expected delay: 77
Oct 18 16:55:16-506711 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 6 , payload 1432 bytes, used on wire 4584 bytes, overhead: 3.201117, expected delay: 651
Oct 18 16:55:18-129071 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 8 , payload 1688 bytes, used on wire 7136 bytes, overhead: 4.227488, expected delay: 19
Oct 18 16:55:19-574095 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 30 , payload 3976 bytes, used on wire 40840 bytes, overhead: 10.271629, expected delay: 59
Oct 18 16:55:20-880147 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 38 , payload 1688 bytes, used on wire 33896 bytes, overhead: 20.080568, expected delay: 22
Oct 18 16:55:23-452704 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 48 , payload 2088 bytes, used on wire 52416 bytes, overhead: 25.103449, expected delay: 22
Oct 18 16:55:23-548806 transport-udp-17165 ERROR Fragmented message sent: fragments needed: 52 , payload 4553 bytes, used on wire 60749 bytes, overhead: 13.342631, expected delay: 45

Up to factor 20x of payload

Matthias Wachs

2012-10-18 17:00

reporter   ~0006454

"expected delay" is the last_expected delay value given to GNUNET_FRAGMENT_context_create

It is initialized with 1000ms by default and then only updated by GNUNET_FRAGMENT_context_destroy

Matthias Wachs

2012-10-18 17:16

reporter   ~0006455

Last edited: 2012-10-18 17:16

 preliminary workaround in rev 24401.
frag_ctx->frag = GNUNET_FRAGMENT_context_create (plugin->env->stats,
              UDP_MTU,
              &plugin->tracker,
==> GNUNET_TIME_relative_max (s->last_expected_delay, GNUNET_TIME_UNIT_SECONDS),
              &udp->header,
              &enqueue_fragment,
              frag_ctx);

Result: Overhead factor < ~ 3, maximum seen 3.2

Now fragmentation overhead is about the overhead sending ACKs

Matthias Wachs

2012-10-19 09:39

reporter   ~0006456

Running the peer over night (17:05 -> 9:30):

mwachs@fulcrum:~/gnunet/gnunet-debug/src/transport$ gnunet-statistics | grep "peers connecte"
          dht # peers connected: 39
     topology # peers connected: 40
          nse # peers connected: 40
         core # peers connected: 40
           fs # peers connected: 39
    transport # peers connected: 42


Fri Oct 19 09:29:47 CEST 2012
    transport # UDP, total, bytes in buffers: 1488
    transport # UDP, total, bytes overhead, sent: 43193099
    transport # UDP, total, bytes payload, sent: 121727162 -> 116.08 MB
    transport # UDP, total, bytes, received: 422542349 -> 402.96 MB
    transport # UDP, total, bytes, sent, success: 165321397 -> 157.66 MB -> 2.73 KB/s AVG
    transport # UDP, total, messages, sent, success: 376893
    transport # UDP, total, msgs in buffers: 2

LRN

2012-10-22 10:01

reporter   ~0006461

Your last fix in r24401 dramatically slowed down udp unreliability transport tests.

Matthias Wachs

2012-10-22 10:30

reporter   ~0006462

0006461: by intention

Matthias Wachs

2012-10-23 10:28

reporter   ~0006472

Fragmentation still floods after changes in fragmentation (rev 24468)

Matthias Wachs

2012-10-23 11:27

reporter   ~0006473

Analysis of problems with revision 24468:

I would describe UDP fragmentation sending behaviour with the term "sledgehammer":
- No slowdown when no ACKs are received
- Massive uncontrolled retransmits

The problem is not slowing down sending when/if ACKs arrive, because the ACKs arrive, but until the first ACKs arrive, UDP send with full force!

Example:

Oct 23 10:55:41-102771 transport-24237 ERROR 0xd01e860: New fragmented message 56615 bytes payload -> 41 fragments, delay between fragments: 1, delay between ACKs: 250
Oct 23 10:55:41-162170 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:41-725358 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:42-293725 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:42-857977 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:43-418759 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:43-987010 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:44-581260 transport-24237 ERROR 0xd01e860: Full round transmitted, enqueue next in 500 ms, delay between fragments: 1ms
Oct 23 10:55:45-516439 transport-24237 ERROR 0xd01e860: Processing ACK: received 2 ACKs, send 2 Fragments
Oct 23 10:55:45-517195 transport-24237 ERROR 0xd01e860: Message full ACKed

=> 420 KB in 4,4 seconds -> 95 KB/s just for one message

Matthias Wachs

2012-10-23 13:22

reporter   ~0006474

Since fragmentation code is not documented: created bug 2596

Matthias Wachs

2012-10-24 09:39

reporter   ~0006485

Last edited: 2012-10-24 09:43

 Running a peer with rev 24468 for 16h30:

mwachs@fulcrum:~/gnunet/gnunet-debug/src/transport$ gnunet-statistics | grep connected
           fs # peers connected: 30
    transport # peers connected: 33
         core # peers connected: 32
          dht # peers connected: 31
     topology # peers connected: 32
          nse # peers connected: 32


    transport # UDP, fragmented msgs, bytes overhead, sent, success: 68674012 -> 65.5 MB
    transport # UDP, fragmented msgs, bytes payload, attempt: 252850454 -> 241 MB
    transport # UDP, fragmented msgs, bytes payload, sent, failure: 2552914
    transport # UDP, fragmented msgs, bytes payload, sent, success: 252690168
    transport # UDP, fragmented msgs, fragments bytes, sent, failure: 1489835
    transport # UDP, fragmented msgs, fragments bytes, sent, success: 322710014
    transport # UDP, fragmented msgs, fragments, sent, failure: 1224
    transport # UDP, fragmented msgs, fragments, sent, success: 250857
    transport # UDP, fragmented msgs, messages, attempt: 35674
    transport # UDP, fragmented msgs, messages, pending: 2
    transport # UDP, fragmented msgs, messages, sent, failure: 5
    transport # UDP, fragmented msgs, messages, sent, success: 35667

    transport # UDP, total, bytes in buffers: 0
    transport # UDP, total, bytes overhead, sent: 82278652 -> 1.385 KB/s avg
    transport # UDP, total, bytes payload, sent: 316645189 -> 5.330 KB/s avg
    transport # UDP, total, bytes, received: 1577123758
    transport # UDP, total, bytes, sent, success: 400269675
    transport # UDP, total, bytes, sent, timeout: 24664
    transport # UDP, total, messages, sent, success: 433463
    transport # UDP, total, messages, sent, timeout: 36
    transport # UDP, total, msgs in buffers: 0

Matthias Wachs

2012-10-24 09:43

reporter   ~0006486

This looks OK ... request for feedback!

Issue History

Date Modified Username Field Change
2012-09-17 15:52 aitap New Issue
2012-09-17 15:52 aitap Status new => assigned
2012-09-17 15:52 aitap Assigned To => Matthias Wachs
2012-09-17 15:52 aitap File Added: gnunet-statistics.1347889453
2012-09-27 11:13 Matthias Wachs Note Added: 0006375
2012-09-27 11:27 Matthias Wachs Note Added: 0006376
2012-09-27 11:28 Matthias Wachs Note Edited: 0006376
2012-09-27 11:41 Matthias Wachs Note Added: 0006377
2012-09-27 16:17 Christian Grothoff Target Version => 0.9.4
2012-09-29 21:22 Christian Grothoff Priority normal => urgent
2012-09-29 21:26 Christian Grothoff Severity minor => major
2012-10-06 20:42 Christian Grothoff Note Added: 0006409
2012-10-16 11:04 Matthias Wachs Relationship added related to 0002589
2012-10-17 18:08 Matthias Wachs Note Added: 0006446
2012-10-17 18:09 Matthias Wachs Note Added: 0006447
2012-10-17 18:10 Matthias Wachs Status assigned => feedback
2012-10-17 22:15 Christian Grothoff Note Added: 0006448
2012-10-18 14:58 Matthias Wachs Note Added: 0006451
2012-10-18 15:58 Matthias Wachs Note Added: 0006452
2012-10-18 16:57 Matthias Wachs Note Added: 0006453
2012-10-18 17:00 Matthias Wachs Note Added: 0006454
2012-10-18 17:16 Matthias Wachs Note Added: 0006455
2012-10-18 17:16 Matthias Wachs Note Edited: 0006455
2012-10-19 09:39 Matthias Wachs Note Added: 0006456
2012-10-22 10:01 LRN Note Added: 0006461
2012-10-22 10:30 Matthias Wachs Note Added: 0006462
2012-10-23 10:28 Matthias Wachs Note Added: 0006472
2012-10-23 11:27 Matthias Wachs Note Added: 0006473
2012-10-23 13:21 Matthias Wachs Relationship added child of 0002596
2012-10-23 13:22 Matthias Wachs Note Added: 0006474
2012-10-23 15:00 Christian Grothoff Relationship deleted child of 0002596
2012-10-24 09:39 Matthias Wachs Note Added: 0006485
2012-10-24 09:42 Matthias Wachs Note Edited: 0006485
2012-10-24 09:43 Matthias Wachs Note Added: 0006486
2012-10-24 09:43 Matthias Wachs Note Edited: 0006485
2012-10-24 15:37 Matthias Wachs Status feedback => resolved
2012-10-24 15:37 Matthias Wachs Resolution open => fixed
2012-11-05 18:33 Christian Grothoff Status resolved => closed
2024-05-03 13:58 Christian Grothoff Category UDP transport => transport service