Server Name Indication (SNI) with Qt and OpenSSL (Not QSslSocket)

If you are reading this post, you are probably looking for a way to implement Server Name Indication (SNI) under Qt. I would even go further and assume that you have been using the QTcpSocket and QSslSocket classes for TCP and encrypted socket communications respectively in your program. You have probably also looked around and cannot find a way to implement SNI in a server program with Qt while using the QSslSocket class.

Let’s stop for a minute and look closely at exactly what we are talking about here. What is SNI and why is it important? Simply put, Server Name Indication is a TLS protocol extension that allows a client to indicate which host name it is requesting. Without SNI, an HTTPS server set up to serve content for multiple host names (virtual hosts) over a unique IP address would not be able to differentiate which client requests are meant for each host name respectively.

During an HTTP communication (request and response), which is not encrypted, the host name can easily be read by a receiving server simply by parsing the request headers. The server then knows whether it needs to respond to let’s say a request for thatPicture.jpg on the website with the host name nachega.com instead of thatPicture.jpg on the website with the host name nicholasnachega.com. The server here is obviously set up to serve contents for both of those websites. Otherwise, it would simply return a response saying that the content is not available.

But what happens if the request is encrypted and sent over an HTTPS communication?

The client and the server then have to do what is called a TLS/SSL handshake first, in which they will exchange a certificate and key and agree on how their communication is going to be encrypted. As during the HTTP communication, the server still needs to know which host name is being requested by the client during the TLS/SSL handshake in order to send the certificate and key that correspond to that host name.

But, unlike during an HTTP communication, the server cannot read the HTTPS request headers before the handshake is complete in order to find out which host name is being requested by the client. Otherwise, reading the request headers before the handshake would be done insecurely without using the certificate and key needed to secure the communication.

It is important to note that knowing the right host name during the TLS /SSL handshake is only important when a server is serving content for multiple host names such as nachega.com and nicholasnachega.com. If a server is only set up to serve content for one host name, let’s say nachega.com, then it would not necessarily have to worry about identifying which host name a client is requesting. It could just simply assume that the requests are meant for nachega.com.

The SNI extension to the TLS protocol adds a way for a client to send the host name to the server during the handshake. This way, the server can know which exact certificate and key it needs to present to the client and complete a successful TLS/SSL handshake. By the time the HTTPS request headers are received by the server, the TLS/SSL handshake will have been successfully completed, and the server will then be able to read the encrypted HTTPS headers and respond to the request.

So, now let’s go back to Qt and SNI. As I wrote earlier, you are probably looking for a way to implement SNI using the QSslSocket class. I am afraid to inform you that, as of the current available version of Qt (5.14.0), you cannot do it using the QSslSocket class.

“Why not?”, you ask. Well, it is simply not implemented yet. QSslSocket does currently allow you to set the certificate and key that will be used during the TLS/SSL handshake, but for a predefined host name. That’s it. As of the day of publication of this post, you do not have access, at least through QSslSocket, to a way that will allow you to access the SNI data sent by the client at the beginning of the TLS/SSL handshake. And without access to the SNI data at the beginning of the TLS/SSL handshake, you simply cannot securely complete a handshake using SNI and QSslSocket.

Obviously, you can successfully complete a TLS/SSL handshake by using QSslSocket alone without relying on SNI. That’s not the issue. You will be limited to setting your server to respond only to requests corresponding to the certificate and key you provided using QSslSocket. The issue arises when you want to be able to set up your server to respond to multiple requests for different host names. You cannot yet set up the certificates for different host names or server names using QSslSocket. That’s the issue.

Now, hold on! You are not completely out of luck. You can still use Qt and SNI in a server program. If you want to continue using QSslSocket, you will have to rewrite the underlying code of how Qt handles the start of TLS/SSL connections. At the moment, Qt uses OpenSSL to handle TLS/SSL connections. If you have enough time and know-how, you can rewrite the QSslSocket class, or extend it, or do whatever you feel like, to support SNI. Not exactly a small task, especially if all you expected was to browse the Internet for an easy solution.

Another way to support SNI in your Qt server program would be to simply not use QSslSocket when you write the part of your program that receives and responds to HTTPS requests. You can use OpenSSL directly to receive requests, decide what happens at the beginning of the TLS/SSL handshake, and then respond. Here is the code for a simple TLS server found on the OpenSSL wiki:

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>
#include <openssl/err.h>

int create_socket(int port)
{
    int s;
    struct sockaddr_in addr;

    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = htonl(INADDR_ANY);

    s = socket(AF_INET, SOCK_STREAM, 0);
    if (s < 0) {
        perror("Unable to create socket");
        exit(EXIT_FAILURE);
    }

    if (bind(s, (struct sockaddr*)&addr, sizeof(addr)) < 0) {
        perror("Unable to bind");
        exit(EXIT_FAILURE);
    }

    if (listen(s, 1) < 0) {
        perror("Unable to listen");
        exit(EXIT_FAILURE);
    }

    return s;
}

void init_openssl()
{ 
    SSL_load_error_strings();   
    OpenSSL_add_ssl_algorithms();
}

void cleanup_openssl()
{
    EVP_cleanup();
}

SSL_CTX *create_context()
{
    const SSL_METHOD *method;
    SSL_CTX *ctx;

    method = SSLv23_server_method();

    ctx = SSL_CTX_new(method);
    if (!ctx) {
        perror("Unable to create SSL context");
        ERR_print_errors_fp(stderr);
        exit(EXIT_FAILURE);
    }

    return ctx;
}

void configure_context(SSL_CTX *ctx)
{
    SSL_CTX_set_ecdh_auto(ctx, 1);

    /* Set the key and cert */
    if (SSL_CTX_use_certificate_file(ctx, "cert.pem", SSL_FILETYPE_PEM) <= 0) {
        ERR_print_errors_fp(stderr);
        exit(EXIT_FAILURE);
    }

    if (SSL_CTX_use_PrivateKey_file(ctx, "key.pem", SSL_FILETYPE_PEM) <= 0 ) {
        ERR_print_errors_fp(stderr);
        exit(EXIT_FAILURE);
    }
}

int main(int argc, char **argv)
{
    int sock;
    SSL_CTX *ctx;

    init_openssl();
    ctx = create_context();

    configure_context(ctx);

    sock = create_socket(4433);

    /* Handle connections */
    while(1) {
        struct sockaddr_in addr;
        uint len = sizeof(addr);
        SSL *ssl;
        const char reply[] = "test\n";

        int client = accept(sock, (struct sockaddr*)&addr, &len);
        if (client < 0) {
            perror("Unable to accept");
            exit(EXIT_FAILURE);
        }

        ssl = SSL_new(ctx);
        SSL_set_fd(ssl, client);

        if (SSL_accept(ssl) <= 0) {
            ERR_print_errors_fp(stderr);
        }
        else {
            SSL_write(ssl, reply, strlen(reply));
        }

        SSL_shutdown(ssl);
        SSL_free(ssl);
        close(client);
    }

    close(sock);
    SSL_CTX_free(ctx);
    cleanup_openssl();
}


You will need to integrate and adapt this code into your Qt server program. “But, where is the SNI part?”, you ask. Well, hold on! We’re getting there. OpenSSL does support Server Name Indication (SNI). As you can read on this thread on the often useful programmers’ resource website StackOverflow:

Set up an additional SSL_CTX() for each different certificate;

  • Add a servername callback to each SSL_CTX() using SSL_CTX_set_tlsext_servername_callback();
  • In the callback, retrieve the client-supplied servername with SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name). Figure out the right SSL_CTX to go with that host name, then switch the SSL object to that SSL_CTX with SSL_set_SSL_CTX().

And that’s it! You now have SNI support in your Qt server program, even if you will have to wait awhile until it is properly added to Qt using QSslSocket.

Could malloc_trim() solve your memory leak?

Memory leaks can be a pain to solve. Software developers spend countless hours debugging their programs to find out which code is causing a memory leak. On Linux systems, there are tools such as Valgrind which can help find the source of a memory leak in a program.

But, sometimes, a program seems to continuously leak memory, even though the programmer has exhausted all debugging techniques, and there seems to be no culprit in the code itself.

One other place to look that is not often mentioned while discussing memory leaks is the operating system itself. Yes, you read that right. The operating system might be the source of that ever-increasing memory load you’re dealing with. While many tutorials focus on the programmer’s code, they often fail to mention that an apparent memory leak can be caused by the operating system allocating memory to a program even when the program has not expressly requested it or does not currently need it.

In order to increase performance, operating systems try to anticipate future memory use from a program. In some cases, a program might request memory from the operating system, and then properly release it back to the operating system when it is no longer needed. But the operating system might keep that memory allocated to the program, anticipating that the program would keep requesting the memory.

The operating system might even allocate more memory to a program than it previously requested. This often happens when a program is creating a lot of objects. And even though the program is properly releasing the memory back to the operating system, the operating system might decide to keep the memory allocated instead of retaking it and having to go through the process of reallocating it to the program later.

One might argue that programs should not create a lot of objects then, in order to avoid causing the operating system to allocate memory that is not needed. But some programs do need to create a lot of objects for them to work properly. So, while both side might share a bit of responsibility in this case, the main point here is that a programmer needs to remember that not all “freed” memory is taken back by the operating system, and that the operating system might even allocate memory in advance.

Thankfully, for Linux systems, there is a way to tell the operating system to trim the heap memory allocated to a program. Once, you’ve made sure that you’ve freed the allocated memory back to the operating system, you can also tell the operating system that you don’t want it to keep the memory around allocated to your program (seemingly against your will). You do this by calling malloc_trim().

The man page for malloc_trim() says that:

“The malloc_trim() function attempts to release free memory at the top of the heap (by calling sbrk(2) with a suitable argument).

The pad argument specifies the amount of free space to leave untrimmed at the top of the heap. If this argument is 0, only the minimum amount of memory is maintained at the top of the heap (i.e., one page or less). A nonzero argument can be used to maintain some trailing space at the top of the heap in order to allow future allocations to be made without having to extend the heap with sbrk(2).”

So, if you are having an apparent memory leak, and can’t find the source, try calling malloc_trim() once in a while in your program.

How to solve no service found for – org.qt-project.qt.mediaplayer

If you deploy a Qt Multimedia project to a Linux distro, you might come accross this error message:

defaultServiceProvider::requestService(): no service found for – “org.qt-project.qt.mediaplayer”

The error occurs when there are Qt plugins that are either missing from the files you deployed, or their dependencies on the distro are not installed.

Qt Multimedia platform uses the Gstreamer multimedia framework. So, the first place to look is to make sure that you install all the Gstreamer files that your Qt Multimedia program might need. There are several libraries available for GStreamer. The Qt documentation says that you need to install even the “ugly”, “good”, “bad”, and ffmpeg and libav libraries.

To build Qt Multimedia, you need the GStreamer library, base plugins, and development files for your system. To run applications that use Qt Multimedia, you might also need to install the following GStreamer plugins: ‘good’, ‘ugly’, ‘bad’, ffmpeg (0.10), and libav (1.x). These additional plugins contain various codecs for audio and video decoding, as well as the necessary components for using the camera APIs. The package names for GStreamer vary between Linux distributions; try searching for gstreamer or libgstreamerin your distribution’s package repository to find suitable packages.

Here is a command to install many of these libraries at once with yum:

yum install gstreamer1-devel gstreamer1-plugins-base-tools gstreamer1-devel-docs gstreamer1-plugins-base-devel gstreamer1-plugins-base-devel-docs gstreamer1-plugins-good gstreamer1-plugins-good-extras gstreamer1-plugins-ugly gstreamer1-plugins-ugly-devel-docs gstreamer1-plugins-bad-free gstreamer1-plugins-bad-free-devel gstreamer1-plugins-bad-free-extras

If you are still getting the error after installing the GStreamer libraries, then you should look at the Qt Multimedia plugins you deployed with your program.

The easiest way to find out if one of your libraries is not being loaded properly by the program is to set the QT_DEBUG_PLUGINS to a non-zero value. This will make the program print debug information about the libraries it is loading during runtime.

You can set the QT_DEBUG_PLUGINS environment variable with the export command:

export QT_DEBUG_PLUGINS=1

You can check to see if the variable has been set with the command:

echo $QT_DEBUG_PLUGINS

Which should print: 1

If you run your program from the terminal, you will now see debug information about the plugins being loaded. If a plugin that is needed was not loaded, the program should print its name and where it looked for the plugin.

In my case, the debug information showed that the “mediaservice” folder was not deployed. After adding it to the deployed files, the debug information showed that the “libqgsttools_p” libraries were not loading properly.

I had to run the command: ldd libgstmediaplayer.so from the mediaservice directory to find out which dependencies were still missing. The libQt5MultimediaWidgets.(*) files and the libqgsttools_p.(*) files were missing from the /lib directory. The mediaservice folder was also missing from the plugins directory.

It turns out that the linuxdeployqt tool I used to package the libraries and create the AppImage file did not include the missing files. You can find the “mediaservice” folder and other missing libraries under your Qt installation. To avoid this error, you need to make sure they are deployed with your program.

After installing the missing Qt Multimedia files, your program should be able to run.