Testing

Tests can be performed using cargo test, only tests of the merc_sabre crate can be executed with cargo test -p merc_sabre --lib and cargo test -- --no-capture can be used to show the output of tests. Alternatively, an improved test runner called nextest can be used with cargo nextest run and can be installed using cargo install cargo-nextest. This test runner offers many improvements such as always showing output of failing tests, running more tests in parallel, and offer better error messages for segmentation faults (although these should be exceedingly rare). Some tests that are ignored by default require a larger stack size, which can be set using the environment variable RUST_MIN_STACK.

LLVM Sanitizer

For Linux targets it is possible to run the LLVM address sanitizer to detect (among others) use-after-free issues in unsafe code. This requires the nightly version of the rust compiler, which can acquired using rustup toolchain install nightly and the rust-src for the standard library, to be installed with rustup component add rust-src --toolchain nightly. To show the symbols for the resulting stacktrace it is also convenient to install llvm-symbolizer, for example using sudo apt install llvm on Ubuntu. Afterwards, the tests can be executed with the address sanitizer enabled using cargo +nightly xtask address-sanitizer. Similarly, we also provide a task for the thread sanitizer and memory sanitizer.

Additional checks and miri

To check for additional undefined behaviour at runtime we can also employ the cargo careful project. It compiles the standard library in nightly with many additional checks for undefined behaviour. It can also be installed with cargo install cargo-careful and requires the nightly toolchain. Then it can be executed with cargo +nightly careful nextest run --target=x86_64-unknown-linux-gnu (or test when nextest has not been installed). There is also a feature merc_debug that enables additional runtime checks. Furthermore, we test with cargo miri for unsafe code violations. This requires the merc_miri feature to be enabled using --features=merc_miri, which is automatically done by cargo xtask miri test.

Code Coverage

Code coverage can be automatically generated for the full workspace using a cargo task. The code coverage itself is generated by LLVM with source code instrumentation, which requires the preview tools to be installed rustup component add llvm-tools-preview, and grcov, which can be acquired using cargo install grcov. To execute the code coverage task use cargo xtask coverage test. The resulting HTML code coverage report can be found in target/coverage or online at the following page.

Benchmarks

Micro-benchmarks can be executed using cargo bench. Additionally, we can also install cargo-criterion and run cargo criterion instead to keep track of more information such as changes over time.

Profiling

Tools built using the release compilation profile automatically contain all the debugging information required for profiling with external tools, such as Intel VTune or perf.

Another useful technique for profiling is to generate a so-called flamegraph, which essentially takes the output of perf and produces a callgraph of time spent over time. These can be generated using the flamegraph-rs tool, which can be acquired using cargo install flamegraph. Note that it relies on either perf or dtrace and as such is only supported on Linux and MacOS.

Finally, in performance critical situations it can be useful to view the generated assembly, which can be achieved with the cargo asm --rust --simplify -p <package> [--lib] <path-to-function> that can be obtained by cargo install cargo-show-asm.