Architecture

The Seele judging system consists of two components: the judging service and the security sandbox. The judging service is a service process that receives user-submitted requests and generates a security sandbox subprocess to run the judging program. The overall architecture of the judging system is shown in the following figure:

The judging service is written in Rust (opens in a new tab) and based on the popular high-performance asynchronous programming framework Tokio (opens in a new tab). The Rust language, with its unique ownership system and memory safety features, allows people to manage memory more safely and efficiently when writing high-performance applications, avoiding application crashes due to memory issues. In addition, Rust also has excellent concurrency performance and scalability, making it an ideal choice for writing high-throughput and high-concurrency performance applications. The Tokio framework brings a high-performance asynchronous runtime implementation to the Rust language, providing an event-driven non-blocking I/O model that allows the judging service to efficiently handle a large number of I/O tasks.

For the security sandbox, it is written in Go (opens in a new tab) and based on the famous container runtime runc (opens in a new tab). See Security Sandbox for details.

Exchange

The Exchange receives judging tasks submitted by users, hands them over to the Composer, and then sends the judging report generated by the Composer back to the user. To improve flexibility, the judging system allows users to configure multiple Exchanges to work together. Currently, the judging service provides two implementations of the Exchange, providing users with multiple ways to interact with the judging system.

Composer

The Composer receives judging tasks from the Exchange, parses the judging tasks, and generates a multi-way tree composed of steps. It sends the steps to the Worker along the tree from the root and tracks the execution of the steps. When no more steps can be executed along the multi-way tree, the judging task is completed. At this point, the Composer summarizes the data to generate the judging report and sends it to the Exchange.

Worker

The Worker receives execution steps sent by the Composer and executes the corresponding tasks according to the configuration. Currently, two types of tasks are provided in the Worker: adding files and running containers. In the running container task, the Worker pulls the image from the user-specified mirror source to the local machine via skopeo (opens in a new tab), and then decompresses the image via umoci (opens in a new tab). The Worker starts the container, runs the judging program, and collects the report by calling the security sandbox program.

Thread Pool

Seele runs Exchange, Composer, and Worker in the main thread to ensure that the concurrency capabilities brought by other CPU cores can be more allocated to the running of the security sandbox, improving the concurrency execution capability of the judging service. At the same time, to ensure that the main thread running the Tokio framework event loop is not affected by synchronous tasks with long running time, causing the event response time to increase and dragging down the overall performance of the system, some blocking I/O tasks and CPU-intensive tasks need to be distributed to the auxiliary threads in the thread pool to run. When running each security sandbox, since it is necessary to ensure that the judging program can occupy a CPU core as much as possible to ensure fairness, we also need to send the task of running the security sandbox to the auxiliary threads in the thread pool.

In addition, since the running time of the security sandbox is much longer than other synchronous tasks, in order to avoid the Tokio thread pool being occupied by security sandbox running tasks and causing other critical synchronous tasks to be unable to be executed in a timely manner, Seele builds a security sandbox thread pool based on the Tokio thread pool. This thread pool only contains some threads in the Tokio thread pool, avoiding the security sandbox running tasks occupying the Tokio thread pool, and the threads in the security sandbox thread pool can still be used by other synchronous tasks, improving the utilization of threads.

When the judging service starts, the main thread and the threads in the Tokio thread pool are bound to different CPU cores of the system through the cpu controller of the cgroup by the judging service. This provides fairness for the running of the security sandbox (reducing the fluctuation of running time) and also improves the running performance of each thread to a certain extent through better CPU cache locality. See Fairness for details.