View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001434 | libmicrohttpd | performance | public | 2008-11-03 17:16 | 2008-11-16 17:00 |
| Reporter | Matthew Moore | Assigned To | Christian Grothoff | ||
| Priority | normal | Severity | feature | Reproducibility | always |
| Status | closed | Resolution | fixed | ||
| Product Version | 0.4.0pre0 | ||||
| Summary | 0001434: Absence of TCP_NDELAY or TCP_CORK can result in performance-degrading interaction with TCP delayed ack | ||||
| Description | Observed this while performance testing with a Windows/Java load-testing client ("The Grinder"). Not setting TCP_NDELAY or TCP_CORK means the Nagle algorithm is engaged in libmicrohttpd. This causes incomplete packets (which can occur because of either a short response body or at the tail end of a larger response) to be held up waiting for an ACK from the client system. Unfortunately under some circumstances the client will wait up to 200ms to send an ACK, hoping it can piggyback the ACK on an outgoing packet (of which there is none with HTTP). As a result I was seeing an extra 200ms of latency on each request. This article describes the issue well: http://www.stuartcheshire.org/papers/NagleDelayedAck/ | ||||
| Additional Information | The TCP_CORK socket option is set while generating an outbound message and unset when the message is complete. It instructs the TCP stack to not send less-than-full packets until TCP_CORK is unset, resulting in optimal network utilization. Unfortunately it is not as portable as TCP_NDELAY which merely disables the Nagle algorithm. Apache sets TCP_NDELAY I believe. I have attached a patch against SVN revision 7820 that sets TCP_CORK while sending a response and unsets it when finished. I added the autoconf stuff to make the use of TCP_CORK conditional. It fixed my performance problem. | ||||
| Tags | No tags attached. | ||||
| Attached Files | tcpcork.patch (2,035 bytes)
Index: configure.ac
===================================================================
--- configure.ac (revision 7846)
+++ configure.ac (working copy)
@@ -136,7 +136,7 @@
AC_CHECK_HEADERS([fcntl.h math.h errno.h limits.h stdio.h locale.h sys/stat.h sys/types.h pthread.h],,AC_MSG_ERROR([Compiling libmicrohttpd requires standard UNIX headers files]))
# Check for optional headers
-AC_CHECK_HEADERS([sys/select.h sys/types.h sys/time.h sys/msg.h netdb.h netinet/in.h time.h sys/socket.h sys/mman.h arpa/inet.h])
+AC_CHECK_HEADERS([sys/select.h sys/types.h sys/time.h sys/msg.h netdb.h netinet/in.h netinet/tcp.h time.h sys/socket.h sys/mman.h arpa/inet.h])
# IPv6
AC_MSG_CHECKING(for IPv6)
@@ -161,6 +161,8 @@
)
AC_MSG_RESULT($have_inet6)
+# TCP_CORK
+AC_CHECK_DECL([TCP_CORK], [], [], [[#include <netinet/tcp.h>]])
# libcurl (required for testing)
SAVE_LIBS=$LIBS
Index: src/daemon/connection.c
===================================================================
--- src/daemon/connection.c (revision 7846)
+++ src/daemon/connection.c (working copy)
@@ -1921,6 +1921,14 @@
continue;
}
connection->state = MHD_CONNECTION_HEADERS_SENDING;
+
+#if HAVE_TCP_CORK
+ /* starting header send, set TCP cork */
+ {
+ const int val = 1;
+ setsockopt(connection->socket_fd, IPPROTO_TCP, TCP_CORK, &val, sizeof(val));
+ }
+#endif
break;
case MHD_CONNECTION_HEADERS_SENDING:
/* no default action */
@@ -1976,6 +1984,13 @@
/* no default action */
break;
case MHD_CONNECTION_FOOTERS_SENT:
+#if HAVE_TCP_CORK
+ /* done sending, uncork */
+ {
+ const int val = 0;
+ setsockopt(connection->socket_fd, IPPROTO_TCP, TCP_CORK, &val, sizeof(val));
+ }
+#endif
MHD_destroy_response (connection->response);
if (connection->daemon->notify_completed != NULL)
connection->daemon->notify_completed (connection->
| ||||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2008-11-03 17:16 | Matthew Moore | New Issue | |
| 2008-11-03 17:16 | Matthew Moore | File Added: tcpcork.patch | |
| 2008-11-08 02:20 | Christian Grothoff | Status | new => assigned |
| 2008-11-08 02:20 | Christian Grothoff | Assigned To | => Christian Grothoff |
| 2008-11-08 02:21 | Christian Grothoff | Status | assigned => resolved |
| 2008-11-08 02:21 | Christian Grothoff | Resolution | open => fixed |
| 2008-11-08 02:21 | Christian Grothoff | Note Added: 0003729 | |
| 2008-11-16 17:00 | Christian Grothoff | Status | resolved => closed |