Search
The main goal of this task is to implement simple server applications in two languages (C/C++/Rust and Java) and optimize its performance in order to be able to handle many clients and their requests.
Minor goals: get familiar with documentation and source code of used platforms, application of the knowledge gained on lectures.
Write an TCP + protobuf based server application, which will count unique words in data sent by clients. The server must meet the following requirements:
You can use any hardware platform and operating system.
Use the following schema for communication with our clients:
syntax = "proto3"; option java_multiple_files = true; option java_package = "cz.cvut.fel.esw.server.proto"; message Request { message GetCount {}; message PostWords { bytes data = 1; } oneof msg { GetCount getCount = 1; PostWords postWords = 2; } } message Response { enum Status { OK = 0; ERROR = 1;}; Status status = 1; // Always present int32 counter = 2; string errMsg = 3; }
You can use any library written in C/C++/Rust (no other languages please).
You can use any library written in Java (you are not allowed to use libraries that are using JNI except libraries that are using functionality implemented in JDK).
At your own risk, you can implement the server in Kotlin (I don't know Kotlin, so you need to make an extra effort for the code to be very readable) or other JVM language, but discuss it with me first.
The performance of server will be measured by a test program, which will send a bigger amount of data, and then ask for the current value of unique words. Total time from the first Request message send to the counter value reception will be measured.
Solutions twice faster than example server below will be accepted. The reference solution is measured on a computer in the laboratory.
Easy server implementation in Python language, which meets the requirements (except performance), is shown below. You can use this implementation as a reference for your solution. Let's suppose that you saved given proto schema as esw_server.proto and generated python files by using protoc --python_out=. esw_server.proto, then you can use the following code.
protoc --python_out=. esw_server.proto
#!/usr/bin/env python3 import asyncio import gzip import struct import esw_server_pb2 words = {} def send_message(writer, message): data = message.SerializeToString() data_size = struct.pack("!I", len(data)) writer.write(data_size + data) async def receive_message(reader, message_type): try: buffer = await reader.readexactly(4) except asyncio.IncompleteReadError as irerr: if len(irerr.partial) > 0: raise ValueError("Message length transmission incomplete") return None data_size, = struct.unpack("!I", buffer) try: data = await reader.readexactly(data_size) except asyncio.IncompleteReadError: raise ValueError("Data transmission incomplete") message = message_type() message.ParseFromString(data) return message async def handle_connection(reader, writer): global words global loop # Client may ask multiple requests while True: request = await receive_message(reader, esw_server_pb2.Request) if request is None: break response = esw_server_pb2.Response() if request.HasField("getCount"): response.status = esw_server_pb2.Response.OK response.counter = len(words) words = {} send_message(writer, response) elif request.HasField("postWords"): text = gzip.decompress(request.postWords.data).decode("utf-8") for word in text.split(): words[word] = 1; response.status = esw_server_pb2.Response.OK send_message(writer, response) else: raise Exception("Request broken") loop = asyncio.get_event_loop() coro = asyncio.start_server(handle_connection, '', 8080, loop=loop) server = loop.run_until_complete(coro) print('Serving on {}'.format(server.sockets[0].getsockname())) loop.run_forever()
You can use the following code to test your server locally. Let say that we saved the following code into tester.py, then we can run python3 tester.py to test trivial functionality of your solution.
python3 tester.py
#!/usr/bin/env python3 import gzip import struct import socket import sys import esw_server_pb2 class Client: def __init__(self, host, port): self.connection = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) self.connection.connect((host, port)) def send_message(self, pb_message): data = pb_message.SerializeToString() data_size = struct.pack("!I", len(data)) self.connection.sendall(data_size + data) def receive_message(self, messageType): data_size, = struct.unpack("!I", self.connection.recv(4)) data = b'' while data_size: buffer = self.connection.recv(data_size) if not buffer: raise ValueError("Data transmission incomplete") data += buffer data_size -= len(buffer) message = messageType() message.ParseFromString(data) return message def get_count(self): request = esw_server_pb2.Request() request.getCount.CopyFrom(esw_server_pb2.Request.GetCount()) self.send_message(request) response = self.receive_message(esw_server_pb2.Response) if response.status == esw_server_pb2.Response.OK: return response.counter else: raise ValueError("Status not OK: " + response.errMsg) def post_words(self, text): request = esw_server_pb2.Request() request.postWords.CopyFrom(esw_server_pb2.Request.PostWords()) request.postWords.data = gzip.compress(text.encode("utf-8")) self.send_message(request) response = self.receive_message(esw_server_pb2.Response) if response.status != esw_server_pb2.Response.OK: raise ValueError("Status not OK: " + response.errMsg) host = sys.argv[1] if len(sys.argv) > 1 else "localhost" port = int(sys.argv[2]) if len(sys.argv) > 2 else 8080 client = Client(host, port) print(client.get_count()) client.post_words("two words") client.post_words("three words words") print(client.get_count())
For a testing use web application, where you put IP address of your server (any public IP, local IPs in KN:E-311 and KN:E-s109 are allowed) and language of your solution. Upload working solutions into upload system as well (code in upload system must be complete with short readme how to compile and run). All your tests must be reproducible. You can be asked for reproduction of similar results during tutorial in order to prove that your solution is correct.
For testing, you can use our sever called Ritchie. You can access it by using ssh username@ritchie.ciirc.cvut.cz with CVUT password. This server is equipped with latest Intel processors running Linux and connected by 10 GBps ethernet to our test server. We also created teams channel dedicated to this server, where you can ask for installation of missing software. Unfortunately, the server is behind firewall, which blocks all outside connections except SSH. Therefore, use the hostname “ritchie.ciirc.cvut.cz” instead of public IP in server configuration in our test web application.
Fastest 8 solutions in each language will gain up to 4 extra points.
The last laboratory is reserved for presentations of the best solutions. Be ready to present your solution. :)
If you have any question, ask in the Server channel in teams.
6 points are for the functionality. There are 3 points for the quality of the code, which are subjective.
For C/C+ +/Rust, subjective evaluation will include but not be limited to: git history, readme and project overview, license, code consistency, readability, and naming conventions.
For Java, the evaluation will focus mostly on code readability and readme.