Introduction
Recently I have introduced us to Cucumber and how to use it in Rust, and while doing the writeup, cucumber-rust 0.7 has been released bringing a huge set of new and unique features. After a closer look through the readme, the strong focus on asynchronous test execution caught my eye. And since I’m a huge fan of ansynchronous programming having done lots of pet stuffs in NodeJS, seeing both my favorite BDD framework and my favorite system level language going strong in async got me severely hyped.
So let’s go! 🙂
Reminder: What is Cucumber?
Cucumber is a framework that implements Behavior Driven Development. The rules of BDD can be summarized as formulizing the requirements step by step in a more and more technical way. We start with the written requirements by your fellow business department and reformulate the requirements into a machine-readable format. Next, we use this text version to write an automated test case that fails, and implement the feature until the test passes. This flow gives it the popular resemblance to Test Driven Development. Cucumber leverages BDD by providing the machine- and human-readable layer based on so-called feature files. These use the Gherkin syntax, a simple syntax based on the keywords Given, When, Then, And and But.
Cucumber is still widely used as a test runner, although BDD is rarely actually applied due to the all-time-popular time limitation in nowaday’s software projects. Another rather unfortunate similarity to TDD.
Reminder: What is Rust?
Rust is a fairly new and rising system level programming language that operates in the same markets as C++ and friends. Besides system-level performance, its main focus lies in builtin security and safety. Furthermore, due to its security and safety-heavy design, it is able to completely omit automated memory management. It just doesn’t need it while still guaranteeing memory safety.
All these points are topped off by an exceptional developer experience: The Rust toolchain brings its full-fledged API documentation and its popular text book right to your command line-operating finger tips, and even compiler errors are designed as tiny educational lessons.
Our Test Object: A Simple AES Encryption Tool
In my previous post, we talked about a small encryption tool with the unspeakable name „Encrsypter“, which was started, when I did my first baby steps in Rust. Today it will serve us once more as our example test object.
The tool is based on aes-gcm, an AES encryption library (or „crate“ in Rust terms) that got audited successfully by the nccgroup. The full source code is available here, but for training purposes, I recommend removing the tests/ directory, as we will incrementally build it up during the tutorial.
Writing Cucumber-based Tests
Before we add the sources for our test cases, let’s check the test object’s project layout. We will start with the following directories and files:
Before we can start coding the test, we must add a cargo-compatible test subproject structure. On your favorite command line, please create the following directories with these terminal commands (all directories relative to the project root):
mkdir tests mkdir tests/features
We will create and store our feature file that specifies the test steps of our Cucumber test in the features/ subdirectory. The step implementation will later go directly to the tests/ directory alongside the central configuration that we will create now. As described in the official documentation, we create a file called cucumber.rs in tests/ with the following content:
mod encrypt_decrypt_steps;
use async_trait::async_trait;
use encrsypter_lib::{decryptor, encryptor};
use std::borrow::Cow;
use std::convert::Infallible;
pub struct EncrsypterTestWorld {
encryptor: encryptor::Encryptor<'static>,
decryptor: decryptor::Decryptor<'static>,
encrypted_base64: String,
decrypt_result: String,
}
#[async_trait(?Send)]
impl cucumber::World for EncrsypterTestWorld {
type Error = Infallible;
// Much more straightforward than the Default Trait before. :)
async fn new() -> Result<Self, Infallible> {
let key = &[1; 32];
let nonce = &[3; 12];
Ok(Self {
encryptor: encryptor::Encryptor {
input: Cow::Borrowed(""),
key,
nonce,
},
decryptor: decryptor::Decryptor {
file_path: "./testfile.txt",
key,
nonce,
},
encrypted_base64: "".to_string(),
decrypt_result: "".to_string(),
})
}
}
fn main() {
// Do any setup you need to do before running the Cucumber runner.
// e.g. setup_some_db_thing()?;
let runner = cucumber::Cucumber::<EncrsypterTestWorld>::new()
.features(&["./tests/features/"])
.steps(encrypt_decrypt_steps::steps());
// You may choose any executor you like (Tokio, async-std, etc)
// You may even have an async main, it doesn't matter. The point is that
// Cucumber is composable. :)
futures::executor::block_on(runner.run());
}
The EncrsypterTestWorld struct contains the mutable instances of our test objects: the encryptor and decryptor that serve to encrypt and decrypt our messages using AES. Further we will maintain special fields to keep track of the test object’s respective outputs. In version 0.7 we have an actual main function that serves as our entry point instead of the cucumber! macro in the previous version. Here we perform the basic configuration that gets our Cucumber test up and running: We…
- … specify the test’s
Worldstruct containing our test objects, … - … tell Cucumber where to find feature files, …
- … declare the module that contains our step implementations and …
- … declare, which asynchronous executor we use to resolve the async step calls.
During this tutorial we use async-std supported by the futures and async-trait package. The latter is necessary to extend traits with asynchronous functionality that is not officially supported as of now (Rust 1.47.0). async-std is by no means set in stone though; you can use tokio or any other asynchronous runner equally well. I’m just much more familiar with async-std and futures.
The next config part is done in the project’s Cargo.toml. Again according to the official documentation, we should specify the dev-dependencies and the [[test]] directive as shown here:
[package]
name = "encrsypter"
version = "0.1.0"
authors = ["Florian Reinhard <me@florianreinhard.de>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
aes-gcm = "0.6.0"
rand = "0.7.3"
[lib]
name = "encrsypter_lib"
path = "src/lib.rs"
[[test]]
name = "cucumber"
harness = false # Allows Cucumber to print output instead of libtest
[dev-dependencies]
cucumber = { package = "cucumber_rust", version = "^0.7.0" }
base64 = "0.12.3"
futures = "0.3.6"
async-trait = "0.1.41"
In terms of dependencies we need the cucumber_rust package to run our tests and the futures and async-trait packages as discussed above.
Then we need the base64 package, because we will work with and do assertions on raw bytes. Although not entirely necessary, it may come in handy for visualisation purposes.
Under [[test]] we give our Cucumber test a name and we route the execution output to stdout to have a nice and tidy output, where we need it.
Alright, the config is done. Now we are ready to specify our first test case. We will encrypt a small „Hello World!“ message, give it a rough sanity check, and then we decrypt it back and hope that the decrypted output matches our input. Under ./tests/features, please create the file encryptor.feature. The containing test specification should roughly look like this:
Feature: Encrypt messages and write them to a file.
Scenario: Encrypt a simple Hello World - message.
Given I have an encryptor initialized with input "Hello World!"
Then I should see "Hello World!" in the encryptor's input field
When I encrypt the encryptor's input
Then testfile.txt exists
And testfile.txt is not empty
When I decrypt testfile.txt
Then the decrypted result should be "Hello World!"This describes, what we want to accomplish: We want to encrypt the string „Hello World!“ and check, whether the output is there and whether it is not completely broken. Then we want to decrypt that output back and check, whether the output is the same as our input message. Next, we have to actually automate this test by implementing the Givens, Whens, Thens and Ands in the feature file.
Step Implementation Files
So far we have told Cucumber, where to find its stuff, and we created a written test specification. Great, we are almost there. The last step is to weave the magic into the Gherkin steps that do the heavy lifting, when Cucumber reads a step in the current feature file. Lets check out the following example step and see, what that means:
.when_async(
"I encrypt the encryptor's input",
t!(|world, _step| {
world.encryptor.write_encrypted();
world
}),
)
This means whenever the Cucumber engine finds a step that matches „When I encrypt the encryptor’s input“ inside the feature file, the code within the closure that is constructed by the builtin t! macro is executed. Here we encrypt some random text.
The t! macro creates a wrapper around the step-implementing closure that extends it with asynchronous and future-driven functionality. It is exclusive to the asnychronous step methods. In the regular non-asynchronous step methods you can use regular closures.
Back to step implementations; regular expressions are usable, too:
.given_regex_async(
r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#,
t!(|mut world, texts_to_encrypt, _step| {
world.encryptor.input = Cow::Owned(texts_to_encrypt[1].to_owned());
world
}),
)This step defines the text that we want to encrypt using the When step from above. Here the text is derived from the feature file by matching the regular expression and its enclosing capture group ([\w\s!]+). The value that was read by the capture group goes to the custom closure parameter after world, in this case called text_to_encrypt. By using the regular expression above, we could have written the steps in our feature file like the following:
Given I have an encryptor initialized with input "Hi I am Floh"
=> encryptor input is "Hi I am Floh"
Given I have an encryptor initialized with input "99 bottles of beer on the wall…"
=> encryptor input is "99 bottles of beer on the wall…"
Given I have an encryptor initialized with input "Your ad here"„
=> encryptor input is "Your ad here
Putting all the knowledge together, here is the sample implementation for our test steps. Please put it into ./tests/encrypt_decrypt_steps.rs (relative to the project root).
use cucumber::{t, Steps};
use std::borrow::Cow;
use std::fs;
use std::path::Path;
pub fn steps() -> Steps<crate::EncrsypterTestWorld> {
let mut builder: Steps<crate::EncrsypterTestWorld> = Steps::new();
builder
.given_regex_async(
r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#,
t!(|mut world, texts_to_encrypt, _step| {
world.encryptor.input = Cow::Owned(texts_to_encrypt[1].to_owned());
world
}),
)
.then_regex_async(
r#"^I should see "([\w\s!]+)" in the encryptor's input field$"#,
t!(|world, expected_texts, _step| {
assert_eq!(expected_texts[1], world.encryptor.input);
world
}),
)
.when_async(
"I encrypt the encryptor's input",
t!(|world, _step| {
world.encryptor.write_encrypted();
world
}),
)
.then_async(
"testfile.txt exists",
t!(|_world, _step| {
let testfile_path = Path::new("./testfile.txt");
assert_eq!(testfile_path.exists(), true);
_world
}),
)
.then_async(
"testfile.txt is not empty",
t!(|mut world, _step| {
let enc_message = fs::read("./testfile.txt").expect("Could not read test file.");
world.encrypted_base64 = base64::encode(&enc_message);
assert_eq!(world.encrypted_base64.len() > (0 as usize), true);
world
}),
)
.when_async(
"I decrypt testfile.txt",
t!(|mut world, _step| {
world.decrypt_result = world.decryptor.read_decrypted();
world
}),
)
.then_regex_async(
r#"^the decrypted result should be "([\w\s!]+)"$"#,
t!(|mut world, expected_texts, _step| {
assert_eq!(expected_texts[1], world.decrypt_result);
world
}),
);
builder
}
Please note that we use raw string literals written in r#...# in order to spare us escaping intentional doublequotes and backslashes.
Now we are ready for the first test run. Please execute the following command in your favorite terminal:
cargo test --test cucumber
If all goes well, it shows us a positive test result:
Conclusion: The All New Cucumber-Rust
The new version line cucumber-rust 0.7 brought a lot of super powers to the tips of our test automation fingers. With asynchronous tests, we are a huge step closer to real test parallelization and thus to less performance headaches, a quite notorious problem in test automation. The default trait got replaced by an intuitive and asynchronous World::new function, which makes working with Worlds much more intuitive, and as a great personal side effect, I got rid of the hassle that the World instance’s lifetime caused me. This helps me immensely to read, write and reason about the code. In future versions we might expect more simplifying changes to make asynchronous testing even more intuitive. For example with the power of procedural macros maybe we will get by without the t! macro ..?
I’m most certainly looking forward to the future versions.
If you are curious about how the test looked like in 0.6, you can find my previous post here. Or if you’d like to know, why I picked up test automation in the first place, feel free to check this one out. And, as mentioned in my original Cucumber Rust article, here is my quick tutorial on how to use Rust Modules.
Have a great day & happy testing! 🙂
EDIT Nov. 2021: A lot of things have been worked on in Cucumber Rust, so I compiled a comprehensive summary about the most crucial changes here. Hope you enjoy it!
