TCP_MD5SIG: An Undocumented Socket Option in Linux

Although the Linux kernel implements RFC 2385 as TCP_MD5SIG socket option, there are no man page entries describing the functionality and the usage, for kernel as well as user-space. The setsockopt() is a straightforward API, however, the prerequisites or constraints put by the RFC makes it a bit tricky to use. It was meant to put a check on authenticity, although it could also be transparently used for data integrity, where the TCP checksum is not good enough!

RFC 2385 talks about “Protection of BGP Sessions via the TCP MD5 Signature Option”. It was proposed way back in 1998 to avoid the BGP from spoof-attacks wherein the attacker can forge the TCP segments. By adding MD5 signature as a TCP Option (Type #19), this spec provides a mechanism to vouch on the authenticity of the sender and data. The MD5 signature is computed using a pre-shared key between the client and the server.

The socket option TCP_MD5SIG saves a mapping of the pre-shared MD5 key against the corresponding peer endpoint. It is mandatory to bind the client to a particular IP and port known to the server. The setsockopt() must be called on the listen socket of the server and the connection socket of the client, before the connect() gets called from client.

The setsockopt() for TCP_MD5SIG saves a mapping of the pre-shared MD5 key against the corresponding peer endpoint represented by tuple . The server may call setsockopt() with TCP_MD5SIG multiple times, one per client. Thus, the server will have multiple mappings for predetermined clients. If the mapping already exists, it is overwritten. Similarly, the client too has to save a mapping corresponding to the server endpoint. The client usually sets this option just ones as one client socket communicates with just one server.

struct tcp_md5sig {
    struct  __kernel_sockaddr_storage tcpm_addr;
    __u16   __tcpm_pad1;
    __u16   tcpm_keylen;
    __u32   __tcpm_pad2;
    __u8    tcpm_key[TCP_MD5SIG_MAXKEYLEN];
};

Similar structure also exists in the user include file. The tcpm_addr can be of type struct sockaddr_in or struct sockaddr_in6, depending on whether the address is IPv4 or IPv6. TCP_MD5SIG_MAXKEYLEN equals 80. tcpm_key is the key in binary format — meaning it need not be ASCII characters necessarily. The tcpm_keylen is the key-length which must be in range from 1 to TCP_MD5SIG_MAXKEYLEN. The pad fields can be left zero.

As mentioned in the beginning, the RFC puts some constraints. Usually, it is not necessary to call a bind() for a client. However, in this case it is mandatory to bind the client to an IP and port. This IP and port is used by the server to map the key. So also it is necessary that the setsockopt() for TCP_MD5SIG is called before connect(), else the MD5 validation will fail and the connection will be dropped. You may also prefer to bind the server socket to a known IP and port although using INADDR_ANY or in6addr_any also works just fine so long as the client gets the reply from the same IP, that it initiated connection with and used for the MD5 key mapping.

Following is the client-side snippet mapping the key to the server endpoint — the server-side code is alike.

    //Sockets Layer Call: bind()
    memset((char *) &cl_addr, 0, sizeof(cl_addr));
    cl_addr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, c6ip, &cl_addr.sin6_addr)<=0)
        error("ERROR on inet_pton");
    cl_addr.sin6_port = htons(atoi(cport));
    if (bind(sockfd, (struct sockaddr *) &cl_addr, 
                sizeof(cl_addr)) < 0)
        error("ERROR on binding");
 
    memset((char *) &serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, s6ip,
                &serv_addr.sin6_addr)<=0)
        error("ERROR on inet_pton");
    serv_addr.sin6_port = htons(atoi(sport));
    memcpy(&md5.tcpm_addr, &serv_addr, sizeof(serv_addr));
    strcpy(md5.tcpm_key, key);
    md5.tcpm_keylen = strlen(key);
    if ((r = setsockopt(sockfd, IPPROTO_TCP, TCP_MD5SIG,
                    &md5, sizeof(md5))) < 0)
        error("listen setsockopt TCP_MD5SIG");

Unlike TCP windowing, in this option, there is no negotiation during the hand-shake — TCP does not allow the “checking and falling back” depending on whether or not an endpoint supports TCP_MD5SIG. The client’s SYN message is also MD5 signed. Hence, if the server does not find the corresponding MD5 key or the MD5 signature does not match, it will drop the SYN message. On the other hand, if the client SYN does not have the MD5 option in TCP header, and the server has been set with at least one mapping (i.e. at least, one call to setsockopt() with TCP_MD5SIG), the server will drop the SYN.

For any message, whose MD5 signature does not validate or the key is not found, the receiver just drops the segment. The sender, in response, may re-transmit. If the retransmitted segment passes the MD5 validation, the communication proceeds; if not, the connection resets on retransmission timeout.

Sample code:

/**
 * server6.c: TCP_MD5SIG
 */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
 
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <netdb.h>
 
#include <arpa/inet.h>
 
#define BUFFERSZ 256
 
struct tcp_md5sig md5;
 
static inline void error(char *msg) {
    perror(msg);
    exit(1);
}
 
int main(int argc, char *argv[]) {
    int sockfd, newsockfd, portno;
    socklen_t clilen;
    char buffer[BUFFERSZ], *key, *s6ip, *sport, *c6ip, *cport;
    struct sockaddr_in6 serv_addr, saddr, cli_addr;
    int n, r;
    char client_addr_ipv6[100];
 
    if (argc != 6) {
        fprintf(stderr, "Usage: %s <server IPv6> <server port> "
                "<client IPv6> <client port> <MD5 key>\n",
                argv[0]);
        return -1;
    }
 
    s6ip = argv[1];
    sport = argv[2];
    c6ip = argv[3];
    cport = argv[4];
    key = argv[5];
 
    printf("\nIPv6 TCP Server (TCP_MD5SIG) ...\n");
 
    //Sockets Layer Call: socket()
    sockfd = socket(AF_INET6, SOCK_STREAM, 0);
    if (sockfd < 0)
        error("ERROR opening socket");
 
    bzero((char *) &serv_addr, sizeof(serv_addr));
    portno = atoi(sport);
    serv_addr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, s6ip, &serv_addr.sin6_addr) <= 0)
        error("ERROR on inet_pton");
    serv_addr.sin6_port = htons(portno);
 
    //Sockets Layer Call: bind()
    if (bind(sockfd, (struct sockaddr *) &serv_addr,
                sizeof(serv_addr)) < 0)
        error("ERROR on binding");
 
    // Client sockaddr_in6 for TCPMD5_SIG
    memset(&saddr, '0', sizeof(saddr));
    saddr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, c6ip, &saddr.sin6_addr)<=0)
        error("ERROR on inet_pton");
    saddr.sin6_port = htons(atoi(cport));
    md5.tcpm_addr = *(struct sockaddr_storage *) &saddr;
    strcpy(md5.tcpm_key, key);
    md5.tcpm_keylen = strlen(key);
    if ((r = setsockopt(sockfd, IPPROTO_TCP, TCP_MD5SIG,
                    &md5, sizeof(md5))) < 0)
        error("listen setsockopt TCP_MD5SIG");
 
    //Sockets Layer Call: listen()
    listen(sockfd, 5);
    clilen = sizeof(cli_addr);
 
    //Sockets Layer Call: accept()
    newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
    if (newsockfd < 0)
        error("ERROR on accept");
 
    //Sockets Layer Call: inet_ntop()
    inet_ntop(AF_INET6, &(cli_addr.sin6_addr),client_addr_ipv6, 100);
    printf("Incoming connection from client having IPv6 address: %s\n",
            client_addr_ipv6);
 
    memset(buffer,0, BUFFERSZ);
 
    //Sockets Layer Call: recv()
    n = recv(newsockfd, buffer, BUFFERSZ-1, 0);
    if (n < 0)
        error("ERROR reading from socket");
 
    printf("Message from client: %s\n", buffer);
 
    //Sockets Layer Call: send()
    n = send(newsockfd, "Server got your message",
            strlen("Server got your message"), 0);
    if (n < 0)
        error("ERROR writing to socket");
 
    //Sockets Layer Call: close()
    close(sockfd);
    close(newsockfd);
 
    return 0;
}
/**
 * client6.c: TCP_MD5SIG
 */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
 
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <netdb.h>
 
#define BUFFERSZ 256
 
static inline void error(char *msg) {
    perror(msg);
    exit(0);
}
 
struct tcp_md5sig md5;
 
int main(int argc, char *argv[]) {
    int sockfd, portno, n, r;
    struct sockaddr_in6 cl_addr, serv_addr;
    struct hostent *server;
    char buffer[BUFFERSZ] = "This is a string from client!";
    char *key, *s6ip, *sport, *c6ip, *cport;
 
    if (argc != 6) {
        fprintf(stderr, "Usage: %s <server IPv6> <server port> "
                "<client IPv6> <client port> <MD5 key>\n",
                argv[0]);
        return -1;
    }
 
    s6ip = argv[1];
    sport = argv[2];
    c6ip = argv[3];
    cport = argv[4];
    key = argv[5];
    printf("\nIPv6 TCP Client Started...\n");
 
    //Sockets Layer Call: socket()
    sockfd = socket(AF_INET6, SOCK_STREAM, 0);
    if (sockfd < 0)
        error("ERROR opening socket");
 
    //Sockets Layer Call: bind()
    memset((char *) &cl_addr, 0, sizeof(cl_addr));
    cl_addr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, c6ip, &cl_addr.sin6_addr)<=0)
        error("ERROR on inet_pton");
    cl_addr.sin6_port = htons(atoi(cport));
    if (bind(sockfd, (struct sockaddr *) &cl_addr, 
                sizeof(cl_addr)) < 0)
        error("ERROR on binding");
 
    memset((char *) &serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin6_family = AF_INET6;
    if(inet_pton(AF_INET6, s6ip,
                &serv_addr.sin6_addr)<=0)
        error("ERROR on inet_pton");
    serv_addr.sin6_port = htons(atoi(sport));
    memcpy(&md5.tcpm_addr, &serv_addr, sizeof(serv_addr));
    strcpy(md5.tcpm_key, key);
    md5.tcpm_keylen = strlen(key);
    if ((r = setsockopt(sockfd, IPPROTO_TCP, TCP_MD5SIG,
                    &md5, sizeof(md5))) < 0)
        error("listen setsockopt TCP_MD5SIG");
 
    //Sockets Layer Call: connect()
    if (connect(sockfd, (struct sockaddr *) &serv_addr, 
                sizeof(serv_addr)) < 0)
        error("ERROR connecting");
 
    //Sockets Layer Call: send()
    n = send(sockfd, buffer, strlen(buffer)+1, 0);
    if (n < 0)
        error("ERROR writing to socket");
 
    memset(buffer, 0, BUFFERSZ);
 
    //Sockets Layer Call: recv()
    n = recv(sockfd, buffer, BUFFERSZ-1, 0);
    if (n < 0)
        error("ERROR reading from socket");
    printf("Message from server: %s\n", buffer);
 
    //Sockets Layer Call: close()
    close(sockfd);
 
    return 0;
}

3 thoughts on “TCP_MD5SIG: An Undocumented Socket Option in Linux

  1. Thanks for the info on this. I’ve run into the challenge that i need tcp-md5 and thanks to your article, i now know that there is acctually support for it in the linux kernel – and i know how to use it. Thanks again.
    Just one question on this – are you aware of any implementation for tcp-ao (RFC 5925) for the linux kernel ?
    Thanks

    Liked by 1 person

    1. Welcome and thanks for the comment.

      I believe there had been experimental patches proposed for TCP-AO, but they are not there in the mainline kernel in today’s date. I am not sure if there are any reliable floating patches for the same that could be used.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s