Effective servers

The main goal of this task is to implement simple server applications in two languages (C/C++ and Java) and optimize its performance in order to be able to handle many clients and their requests.

Minor goals: get familiar with documentation and source code of used platforms, application of the knowledge gained on lectures.

Write an HTTP 1.1 protocol based server application, which will count unique words in data sent by clients. The server must meet the following requirements:

• Clients send data using POST method with path /esw/myserver/data. Data are in plain text format encoded in UTF-8 and compressed by gzip method.
• The server counts unique words. On startup, the counter equals zero.
• The server keeps records of words sent by clients. The unique word counter is increased by one for each new word, which is not in records yet.
• If a GET request with path /esw/myserver/count arrives, the server will answer actual value of unique count and reset it. The value is transferred as a decadic number in UTF-8 encoding.
• The server must be able to handle a large number of simultaneously connected clients (approximately 100).

You can use any hardware platform and operating system.

You can use any library written in C/C++ (no other languages please).

You can use any library written in Java (you are not allowed to use libraries which are using JNI except libraries which are using functionality implemented in JDK).

Let us suppose that our server runs on local computer (localhost) on port 8080. We will use curl program for sending requests to the server:

curl localhost:8080/esw/myserver/data --data-binary @<(echo first attempt|gzip)
curl localhost:8080/esw/myserver/data --data-binary @<(echo second attempt|gzip)
curl localhost:8080/esw/myserver/count --output -

The output of the last command is “3”. You can use a web browser with address http://localhost:8080/esw/myserver/count to get current counter value instead of the last command.

Note: Syntax <(cmd…) is process substitution in the bash interpreter. It is one of the possible ways how to pass binary data in the command line.

The performance of server will be measured by a test program, which will send a bigger amount of data, and then ask for the current value of unique words. Total time from the first POST request send to the counter value reception will be measured.

Solutions twice faster than example server below will be accepted. The reference solution is measured on a computer in the laboratory.

Easy server implementation in Python language, which meets the requirements (except performance), is shown below. You can use this implementation as a reference for your solution.

#!/usr/bin/python3

from http.server import HTTPServer, BaseHTTPRequestHandler
import gzip

words = {}

class OSPHTTPHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        global words
        if self.path == "/esw/myserver/data":
            length = int(self.headers.get('Content-Length'))
            text = gzip.decompress(self.rfile.read(length)).decode("utf-8")
            for word in text.split():
                words[word] = 1;
            self.send_response(204) # No Content
            self.end_headers()
        else:
            self.send_response(404)
            self.end_headers()

    def do_GET(self):
        global words
        if self.path == "/esw/myserver/count":
            self.send_response(200)
            self.send_header("Content-type", "text/plain")
            self.end_headers()
            self.wfile.write(str(len(words)).encode())
            words = {}
        else:
            self.send_response(404)
            self.end_headers()

class HTTPServerIPv6(HTTPServer):
    import socket
    address_family = socket.AF_INET6

# httpd = HTTPServer(('', 8080), OSPHTTPHandler) # Use if your system uses IPv4 by default
httpd = HTTPServerIPv6(('', 8080), OSPHTTPHandler) # Use if your system uses IPv6 by default
print("Listening on port", httpd.server_port)
httpd.serve_forever()

If you connect too many clients to this server, the server will be overloaded, and some clients will be refused.

For a testing use web application, where you put IP address of your server (any public IP, local IPs in KN:E-311 and KN:E-s109 are allowed) and language of your solution. Upload working solutions into upload system as well (code in upload system must be complete with short readme how to compile and run). All your tests must be reproducible. You can be asked for reproduction of similar results during tutorial in order to prove that your solution is correct.

For testing, you can use our sever called Ritchie. You can access it by using ssh username@ritchie.ciirc.cvut.cz with CVUT password. This server is equipped with latest Intel processors running Linux and connected by 10 GBps ethernet to our test server. We also created a thread in our forum dedicated to this server, where you can ask for installation of missing software. Unfortunately, the server is behind firewall, which blocks all outside connections except SSH. Therefore, use the hostname “ritchie.ciirc.cvut.cz” instead of public IP in server configuration in our test web application.

First 12 solutions in each language will gain up to 4 extra points. Deadline of the contest is set to 20. 5. 2018

The last laboratory is reserved for presentations of the best solutions. Be ready to present your solution. :)

If you have any question, ask in the forum. Your question can be useful for others.