Character array is missing „e“ notation exponential mark

I observed that many people out there visiting my blog have great interest in a particular NumberFormatException. It goes by the message Character array is missing „e“ notation exponential mark and was mentioned in my post about fuzzing in Java.

Today we will take a more in depth look at it.

How does that NumberFormatException look like?

To visualize the error effectively, let’s look at the following extreme example drawn from my fuzzing post mentioned above.

Consider the following (sub-par implemented) function dollar2euro that takes any input (hopefully it’s a number though!) and tries to convert it from USD to EUR:

public String dollar2euro(Object input){
    BigDecimal inputParsed = new BigDecimal(input.toString());
    BigDecimal dollars = inputParsed.setScale(2, BigDecimal.ROUND_HALF_EVEN);

    BigDecimal multiply = dollars.multiply(BigDecimal.valueOf(0.92));
    BigDecimal euros = multiply.setScale(2, BigDecimal.ROUND_HALF_EVEN);
    return String.valueOf(euros);
}

Now what happens, if we play along the method signature and put in the following characters: „뤇皽“? If you know what they mean, feel free to drop me a comment down below. Please note that this is exactly what my fuzztest tried to do. And of course it drops a heavy NumberFormatException upon us:

Input: 뤇皽
java.lang.NumberFormatException: Character 뤇 is neither a decimal digit number, decimal point, nor "e" notation exponential mark.

And what does it mean?

The problem here is that „뤇皽“ is not a number. The first and second part Character 뤇 is neither a decimal digit number, decimal point are straight forward. But what does the third part nor „e“ notation exponential mark mean?

In maths, we have the option of expressing numbers in power form with base 10. This comes in handy when we want to express Googol (the number that inspired Google’s name) – a 1 with 100 zeroes. Accordingly we would write 10100 instead. In Java you can do something very similar, but there are better sources to check how to properly use the e-format. For our context let’s acknowledge that it is a different way of displaying numbers. Which likewise could not be found in our input.

But how do we fix that?

We have to make sure that we provide a String to the BigDecimal Constructor that looks like a valid number – if we want to provide a String at all. An actual number like a double or int would be even better. That would cost us no more than a change in the method’s signature and a small adaption to line 2. If we really want to provide a String, we still can do that, but then we have to make sure that it is properly formatted. A valid input example would be: „1337.012342„.

If you apply this simple rule, you should be spared from ‚Character array is missing „e“ notation exponential mark‘ errors in the future.

Conclusion

So long! I hope this little post gave you an idea about what the error message ‚Character array is missing „e“ notation exponential mark‘ means. Of course this case was kinda constructed and you probably have a less explicit case. If so, feel free to post it in the comments down below and we see what we can do. But for demonstration purposes it should have provided a clear image of what is going on in your program.

As my linked post about fuzzing in Java implies, I’m an avid test automation person. If you want more about automating tests with Java, check out my introduction tutorial about Selenium in Java. If you are more of the Python person, no problem. Here’s the same Selenium tutorial in Python. And if Python and Java are both too mainstream for you, I recommend my tutorial about Cucumber in Rust .

Happy test automating & have a nice day!

Home » Testautomation
Share it with:

Introduction to Web Test Automation – Java Edition

The internet is an incredible market place hosting millions and millions of software products to make our lives indefinitely easier. To keep up with each other, software grew more and more elaborate to provide us customers with lots of different features and services to finally make us purchase a thing or two. Now due to the resulting increase of complexity we must take special care not to break existing features while developing the next big version brimming full of features. Testing that the next code increment does not break existing stuff is what we testers call „regression testing“, and there is one especially economical way of doing that: test automation.

That’s what we’re going to do today. We gonna test and we gonna automate and we do all that in Java this time. Because everyone knows about Java, right? So let’s write once and run everywhere. Given we have a browser there.

Shopping list

To start our test automation journey – again -, we will work with the following tools. The versions are the ones I used at the time of writing. I might update them in the future.

After downloading and installing these tools from their respective web sites, we’re ready to start our system under test.

Deploying the system under test

To achieve this, we simply run the following command:

docker run --name spree --rm -d -p 3000:3000 spreecommerce/spree:3.6.4

If it went well, a quick glance at http://localhost:3000/ should show a catalog page full of cute Rails merch. Now we will install and prepare Selenium.

Geckodriver installation and management

Contrary to the Python version, we use WebDriverManager to free us from the installation hassle of Geckodriver, hence we are done with this step. The tip was sent to me by my good friend Sho. Thank you, mate. I owe you one! 🙂

Initializing our test automation project

Given a successful Gradle installation as linked in the shopping list we can now create our test project. Please open a terminal in a root directory of your choice and do:

mkdir jata_tutorial && cd jata_tutorial
gradle init

When prompted answer with 1 for a basic project and then 2 for Kotlin as our Gradle DSL.

Of course we will use the new APIs as well, since we love to live on the edge here. Please choose „yes“ here.

Finally, when asked, give your project the default name that is the project root directory’s name. Now your boilerplate project is set up and ready to go.

Now it’s time to define our dependencies. In build.gradle.kts please introduce the following lines:

plugins {
    java
}

group = "your.groupid"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.slf4j:slf4j-api")
    implementation("ch.qos.logback:logback-classic:1.4.5")
    implementation("org.apache.commons:commons-lang3:3.12.0")

    testImplementation("org.seleniumhq.selenium:selenium-java:4.7.2")
    testImplementation("io.github.bonigarcia:webdrivermanager:5.3.1")
    testImplementation(platform("org.junit:junit-bom:5.9.1"))
    testImplementation("org.junit.jupiter:junit-jupiter:5.9.1")
}

tasks.getByName<Test>("test") {
    useJUnitPlatform()
}

This sets up our project with some basic metadata like our groupId and a (somewhat) meaningful version. We defined to use Java as our JVM language and JUnit as our test runner. Additionally we will have access to Maven’s central repository, where Gradle can download our project dependencies that we defined in the section of that exact same name.

With these dependencies, we are able to:

  • automagically download and install webdriver executables for Selenium (in our case: Geckodriver)
  • perform Selenium actions including opening and closing browser windows, enter URLs and clicking stuffs
  • generate random test input strings
  • log things to the console in a nice timestamped format (without being scared of Log4Shell)
  • and do all that in a modern JUnit5 – runner

Everything that we need to do very soon.

The boilerplate

Next up, we create the skeleton of our first test class. In src/test/java, please create a file named CrossTests.java containing the following code. Please note that .java source files should be named after the contained class. Here: CrossTests.

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebDriver;

class CrossTests {

    WebDriver driver;

    @BeforeAll
    static void setupClass() {
        
    }

    @BeforeEach
    void setup() {

    }

    @AfterEach
    void teardown() {

    }

    @Test
    void isOpen() {

    }
}

Great. That looks almost like a real test already. Finally we are going to open a browser window.

Starting and closing the browser

In setupClass we call the WebDriverManager and let it work its Geckodriver-initializing magic.

@BeforeAll
static void setupClass() {
    WebDriverManager.firefoxdriver().setup();
}

Then we ensure that a browser is opened before each of the individual tests marked with @Test:

@BeforeEach
void setup() {
    this.driver = new FirefoxDriver();
    this.driver.manage().timeouts().implicitlyWait(Duration.of(5, ChronoUnit.SECONDS));
}

This opens a Firefox window whenever we execute one of the class‘ test methods. This is a very pure and uncustomised instance; we could configure it further if we wanted. But for our purpose at the moment, it is perfectly fine.

Now before we perform any actions on the page, we must make sure that the browser window is closed appropriately after each test run. In teardown, we do:

    @AfterEach
    void teardown() {
        this.driver.quit();
    }

Caveat: confusion potential

Selenium’s WebDriver class has a close()-method, but please always use quit() instead. It performs a clean browser termination with regards to Selenium’s execution flow, whereas close() will just terminate the browser causing you lots of warnings in the log.

Alright! We have taken care of the browser handling. Time to do stuffs with it.

Opening the landing page

The most basic action ist to open a web page. Thankfully this task is handled by Selenium with a simple one-liner. Given that our sample web shop is up and running, we do:

this.driver.get("http://localhost:3000/");

We introduce the line to the test case method isOpen, which we introduced earlier in our boiler plate. But before we head into executing the test case, we have to be aware of the fact: We as the user can see the page being open, but the computer cannot. We have to programmatically verify that our expectation „landing page is open“ is met.

Verifying the outcome

To do that we verify these 3 conditions:

  • the window title contains the landing page’s HTML title
  • the most prominent logo exists on the web page and
  • it’s actually visible (i.e. it does not have its display property set to none).

We add this code directly below the line that opens the website:

assertTrue(this.driver.getTitle().contains("Spree Demo"),
           "'Spree Demo' is not in the browser's title.");

WebElement logoElement = this.driver.findElement(By.id("logo"));
assertTrue(logoElement.isDisplayed());

The first line peeks into the page title, which is another staple of Selenium, and then checks if it contains „Spree Demo“. Using JUnit’s assertTrue, we verify that this is the case, and if not, we print a custom error message. Condition 1 done.

The second line depicts the method you will probably use the most whilst moving forward in your test automation career: Webdriver#findElement goes through the page’s DOM and grabs the element that meets the given criteria. Here we search for an element with the HTML id „logo“. If it fails to find the element, it drops a NoSuchElementError. Therefore, we just implicitly verified condition 2.

The return value is a WebElement object that we can do clicks, inputs and various other actions on. We will use the object’s method isDisplayed() to check if the logo is visible therefore verifying our 3rd condition.

Straight-forward so far. At this point your code should look like that:

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

import java.time.Duration;
import java.time.temporal.ChronoUnit;

import static org.junit.jupiter.api.Assertions.*;

class CrossTests {

    WebDriver driver;

    @BeforeAll
    static void setupClass() {
        WebDriverManager.firefoxdriver().setup();
    }

    @BeforeEach
    void setup() {
        this.driver = new FirefoxDriver();
        this.driver.manage().timeouts().implicitlyWait(Duration.of(5, ChronoUnit.SECONDS));
    }

    @AfterEach
    void teardown() {
        this.driver.quit();
    }

    @Test
    void isOpen() {
        this.driver.get("http://localhost:3000/");
        assertTrue(this.driver.getTitle().contains("Spree Demo"),
                "'Spree Demo' is not in the browser's title.");

        WebElement logoElement = this.driver.findElement(By.id("logo"));
        assertTrue(logoElement.isDisplayed());
    }

}

Let’s execute it. In a terminal window, please switch to the project root and do:

./gradlew test

If everything went well, your console should display a big OK:

BUILD SUCCESSFUL in 7s
4 actionable tasks: 2 executed, 2 up-to-date

Congratulations, you just executed your first Java-Selenium-test:
Our shop’s landing page can be opened and it is displayed correctly.

What is this driver thingy by the way?

Simply put, the driver – in our case Geckodriver – is the glue between our code and Firefox. The same would be valid for Chrome with Chromedriver respectively. This means it is the interface to our browser handling all the requests that we perform :

  • findElement
  • click
  • „are you displayed?“
  • perform key strokes
  • „grab that text of an element“
  • etc. pp.

Under the hood, drivers are http server applications that receive requests sent by Selenium everytime we perform an action similar to these mentioned above. The driver then interacts with the browser in the way we tasked it to and sends a JSON-based response that we can base our future work on. Everything is encapsulated by Selenium so that we can work with simple objects during automating our tests without having to deal with HTTP or JSON.

Performing and validating browser actions

Now that we know the basics of how to write automated tests with Java and Selenium, let’s develop a more complex test.

The best candidates for automation are cross tests. These are tests that cover representative cross sections of the product. One example could be: Put items into the cart, view the cart page and perform the checkout. It is a great practise to start test automation for new projects by coding these cross tests first.

For the sake of brevity here, we will write up the test until reaching the checkout page. The rest is homework. Don’t worry, I got you covered with the full source code in my bitbucket repo.

Ok, without further ado, here’s the code. Please add the following method to the test class.

@Test
void findSpreeBagAndCheckout()throws InterruptedException{
    this.driver.get("http://localhost:3000/");

    // Landing and catalog page
    this.driver.findElement(By.cssSelector("[href='/t/spree']")).click();
    this.driver.findElement(By.cssSelector("a[href*='spree-bag']")).click();

    // Product page

    this.driver.findElement(By.cssSelector("input#quantity")).clear();
    this.driver.findElement(By.cssSelector("input#quantity")).sendKeys("3");
    this.driver.findElement(By.id("add-to-cart-button")).click();

    // Cart page
    assertTrue(this.driver.getTitle().contains("Shopping Cart"),"'Shopping Cart' is not in the browser's title.");
    this.driver.findElement(By.cssSelector("img[src$='/spree_bag.jpeg']")).isDisplayed();
    String qtyValue=this.driver.findElement(By.cssSelector(".line_item_quantity")).getAttribute("value");
    assertEquals(qtyValue,"3","The item quantity is not the same we typed in!");

    String cartTotal=this.driver.findElement(By.cssSelector(".cart-total > .lead")).getText();
    assertEquals(cartTotal,"$68.97","The total cart value does not match our expectations. That's a blocker!");
}

What’s new here?

If we look back at our first landing page test, you will notice that we use a lot of new things. We will cover them one by one in the next section.

CSS selectors

One thing that catches the eye is that we looked for elements by using CSS expressions. You may have heard about them from your frontend peers, or maybe you have even worked with them yourself already. If not, don’t worry. We will cover them in a followup post. For now, I will give you a few translations:

[href='/t/spree']

„Give me a page element, whose attribute href is equal to ‚/t/spree'“. Quick reminder: href is used in link elements for defining the target URL that is opened, when the user clicks the link.

a[href*='spree-bag']

„Give me any link („anchor“) element, whose attribute href contains ’spree-bag‘.“

img[src$='/spree_bag.jpeg']

„Give me an image element, whose attribute src ends with ‚/spree_bag.jpg‘.“

input#quantity

„Give me an input element, whose HTML id is equal to ‚quantity‘.“ Yes, we could have used By.ID with „quantity“ here, but in this case I wanted to be explicit about the element type input.

.line_item_quantity

„Give me an element that has line_item_quantity in its class attribute.“ The leading dot indicates that we are looking for elements that has the specified class in its class list. It is roughly equivalent to [class*='line_item_quantity'].

.cart-total > .lead

„Give me any element that has the lead class and that’s preceded by any element that has the cart-total class.“

By using CSS selectors you can be very specific about what element you want to use in your test case. This makes CSS selectors incredibly powerful and versatile while maintaining a solid degree of simplicity, especially compared to XPath (ugh).

New properties and methods used in our test case

Aside from CSS selectors, we used several new methods and properties in our automated test.

click() issues a click on the target element. Straight-forward.

.getText() returns the element’s written text. In <p>Some text</p> for example, we would get access to „Some text“.

getAttribute("someHtmlAttribute") gives us the content of an element’s HTML attribute. In our test we fetch an element’s value attribute that is often used to set the content of an input element. Another example might be the link element we talked about earlier. Given we want to have its href attribute for some further verifications; on our page:

<a id="mylink" href="https://www.example.org">an example link</a>

Then in our test we could do:

targetHref = this.driver.findElement(By.id("mylink"))
              .getAttribute("href");
this.doSomeStuffsOn(targetHref);

.sendKeys("My desired input") imitates key strokes on an element. We use it to type any string we want into our target text input element. In our test for example that would be the „quantity“ input box.

And finally, with clear() we have a little helper in our toolbox that removes any content from an input element. In our test we use that to remove the preset value. Otherwise we would accidentally order 31 bags: „3“ from us and „1“ from the value preset by the page.

Continuing your test automation journey from here

Alright, let’s sum up what we accomplished:

We have seen how to open and close a browser window, do various actions on the page and verify their outcomes. Finally, we used that to write a fairly large cross test, all of that on a full fledged ecommerce web app. Great job! Now where should we go next? For the next post, as promised, I’d like to go back one step and give you a deeper introduction to CSS expressions, because they will accompany you for a long time during your test automation journey. Afterwards we have a big problem to solve that you probably already noticed:

Copious amounts of repetition.

To tackle that we will leverage cucumber-jvm, a Behavior Driven Development framework that makes it possible to write test cases in human-readable text form on top of a DRY Java code base. Afterwards we will apply the Page Object Pattern, a test automation – specific design pattern that reduces repetition even more.

Okay! To keep you busy while I am busy, how about finishing the checkout? As a reminder, you can find the full source code in my bitbucket repo at any time. Alternatively you can spoil yourself a little about Cucumber in Rust or try Python instead of Java.

Stay curious and see you in the next post!

Home » Testautomation
Share it with:

Introduction to Web Test Automation in Python

The internet is an incredible market place hosting millions and millions of software products to make our lives indefinitely easier. To keep up with each other, software grew more and more elaborate in order to provide us customers with lots of different features and services to make us purchase a thing or two. Now due to the resulting increase of complexity we must take special care not to break existing features while developing the next big version. Testing that the next code increment does not break existing stuff is what we testers call „regression testing“, and there is one especially economical way of doing that: test automation.

That’s what we’re going to do today. We gonna test and we gonna automate and we do all that in Python, because with its well-designed concepts Python is suitable for beginners and experts alike. So let’s start!

Shopping list

To start our test automation journey, we will work with the following tools. The versions are the ones I used at the time of writing. I might update them in the future.

  • Docker, preferably in its latest version
  • Python 3 version 3.10.8
  • Firefox Browser 106.0.1
  • geckodriver for Firefox 0.32.0
  • a Spree Commerce docker image version 3.6.4 as our system under test
    (will be pulled automagically in the next step)
  • Selenium for Python 4.5.0
  • unittest as our first test runner (comes with Python 3)

After downloading and installing Docker and Python from their respective web sites, we’re ready to start our system under test.

Deploying the system under test

To achieve this, we simply run the following command:

docker run --name spree --rm -d -p 3000:3000 spreecommerce/spree:3.6.4

If it went well, a quick glance at http://localhost:3000/ should show a shopping catalog page full of cute Rails merch. Now we will prepare and install Selenium.

Installing Geckodriver and Selenium

First we need to download geckodriver that will be responsible for sending our actions to the browser. Please see our shopping list for a download page. Download and extract the appropriate archive and put the extracted executable on your PATH. Please do that in a way that fits your OS and your preferences. I’m a mac user, therefore I move it to /usr/local/bin:

$ mv ~/Downloads/geckodriver /usr/local/bin/ 

If everything went well,

$ geckodriver -h 

should print the version and an instruction page. If not, feel free to drop me a Q in the comments below. Otherwise we move on to installing Selenium.

We will leverage pip3 to quickly go through that step:

pip3 install selenium==4.5.0

Now we’re all set. Let’s go headlong into the real trouble.

Test runner boilerplate

First of all, we create the skeleton of our test class. We write a simple Python unittest class that we will fill in with Selenium web test automation goodness as we go further. Our goal is to see the basic structure of a unittest-based automated test case that we will leverage to execute our browser magic. In your favorite project directory, please create a file named cross_tests.py and fill it with the following code. We will talk about the file name a bit later. Don’t worry, it was not a technical decision.

import unittest

# Start with a base class and derive it from 
# unittest.TestCase
class TestSpreeShop(unittest.TestCase):

    # setUp is executed at the start...
    def setUp(self):
        pass

    # ... and tearDown at the end of each test case.
    def tearDown(self):
        pass

    # This will be our first test:
    # We will open the shop's homepage
    # and verify that it worked.
    # For now, we will just let it pass.
    def test_isOpen(self):
        pass


# The main entry point of our unittest-based execution 
# script. Think of it as an actual main method similar 
# to Java's or C's.
if __name__ == '__main__':
    unittest.main()

Great, that almost looks like a real test already. Now finally we are going to open a browser window.

Starting and closing the browser

We write a small support function:

def open_browser(self):
    return webdriver.Firefox()

And we apply it in setUp:

def setUp(self):
    self.driver = self.open_browser()

This opens a Firefox window it its most pure form whenever we execute one of the class‘ test methods. We could configure it further in open_browser, but for our purpose at the moment, this is perfectly fine.

Now before we perform any actions on the page, we must make sure that the browser window is closed appropriately after each test run. In tearDown, we do:

def tearDown(self):
    self.driver.quit()

Caveat: confusion potential

Selenium’s webdriver has a close()-method, but please always use quit() instead. It takes care of clean browser termination with regards to Selenium’s execution, whereas close() will just terminate the browser causing warnings.

Alright! We have taken care of the browser handling. Time to do stuffs with it.

Opening the landing page

The most basic action ist to open a web page. Thankfully this task is handled by Selenium with a simple one-liner. Given our sample web shop is up and running, we do:

self.driver.get("http://localhost:3000/")

We introduce the line to the test case method test_isOpen, which we introduced in our boiler plate. But before we head into executing the test case, we have to be aware of the fact that we as the user can see the page being open, but the computer can’t. We have to programmatically verify that our expectation „landing page is open“ is met.

Verifying the outcome

To do that we verify these 3 conditions:

  • the window title contains the landing page’s HTML title
  • the most prominent logo exists on the web page and
  • it’s actually visible (i.e. it does not have its display property set to none).

We add this code directly below the get – line from before:

self.assertTrue("Spree Demo" in self.driver.title, 
                "'Spree Demo' is not in the browser's title.")

logo_element = self.driver.find_element(By.ID, "logo")
self.assertTrue(logo_element.is_displayed())

The first line peeks into the page title, which is another staple of Selenium, and then checks, if it contains „Spree Demo“ with Python’s in-operator. Using unittest's assertTrue, we verify that this is the case, and if not, we print a custom error message. Condition 1 done.

The second line depicts the method you will probably use the most whilst moving forward in your test automation career: find_element goes through the page’s DOM and grabs the element that meets the given criteria. Here we search for an element with the HTML id „logo“. If it fails to find the element, it drops a NoSuchElementError. Therefore, we just implicitly verified condition 2.

The return value is a WebElement object that we can do clicks, inputs and various other actions on. We will use the object’s method is_displayed() to check, if the logo is visible therefore verifying our 3rd condition.

Straight-forward so far. At this point your code should look like that:

import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By

# Start with a base class and derive it from unittest.TestCase
class TestSpreeShop(unittest.TestCase):

    # setUp is executed at the start...
    def setUp(self):
        self.driver = self.open_browser()

    # ... and tearDown at the end of each test case.
    def tearDown(self):
        self.driver.quit()

    def test_isOpen(self):
        self.driver.get("http://localhost:3000/")
        self.assertTrue("Spree Demo" in self.driver.title, 
                        "'Spree Demo' is not in the browser's title.")

        logo_element = self.driver.find_element(By.ID, "logo")
        self.assertTrue(logo_element.is_displayed())


    # Support method. We will use it to open a browser instance in setUp.
    def open_browser(self):
        return webdriver.Firefox()

# The main entry point of our unittest-based test execution script.
# Think of it as an actual main method similar to Java's or C's.
if __name__ == '__main__':
    unittest.main()

Let’s execute it by doing:

python3 cross_tests.py # Linux or MacOS
python cross_tests.py # Windows

If all went well, your console should display a big OK:

➜  pyta_tutorial git:(main) python3 cross_tests.py
.
----------------------------------------------------------------------
Ran 1 test in 4.719s

OK

Congratulations, you just executed your first Python-Selenium-test:
Our shop’s landing page can be opened and it is displayed correctly.

What is this driver thingy by the way?

Simply put, the driver is the glue between our code and Gecko (or Chromedriver respectively), thus it’s the interface to our browser window. It handles all the requests that we perform like find_element, click, „are you displayed?“, perform key strokes, „grab that text of an element“ etc. pp. Under the hood, Gecko is an http server, which receives requests sent by the driver everytime we perform one of the above actions. The driver then interacts with the browser in the way we tasked it to and sends a JSON-based response object that we can base our future work on. Everything is encapsulated by Selenium so that we can work with simple objects during test automation development without having to deal with HTTP or JSON.

Performing and validating browser actions

Now that we know the basics of how to write automated tests with Python and Selenium, let’s develop a more complex example.

The best candidates for automated tests are cross tests. These are tests that cover representative cross sections of the product. One example could be: Put items into the cart, view the cart page and perform the checkout. This is the reason we named our test file cross_tests.py. It is good practise to start test automation for a new project by coding these cross tests first.

For the sake of brevity here, we will write up the test until the checkout page. The rest is homework. Don’t worry, I got you covered with the full source code in my bitbucket.

Ok, without further ado, here’s the code. Please add the following method to the test class:

def test_findSpreeBagAndCheckout(self):
    self.driver.get("http://localhost:3000/")

    # Landing and catalog page
    self.driver.find_element(By.CSS_SELECTOR, "[href='/t/spree']").click()
    self.driver.find_element(By.CSS_SELECTOR, "a[href*='spree-bag']").click()

    # Product page
    self.driver.find_element(By.CSS_SELECTOR, "input#quantity").clear()
    self.driver.find_element(By.CSS_SELECTOR, "input#quantity").send_keys("3")
    self.driver.find_element(By.ID, "add-to-cart-button").click()
        
    # Cart page
    self.assertTrue("Shopping Cart" in self.driver.title, 
                  "'Shopping Cart' is not in the browser's title.")
    self.driver.find_element(By.CSS_SELECTOR, "img[src$='/spree_bag.jpeg']").is_displayed()
        
    qty_value = self.driver.find_element(By.CSS_SELECTOR, ".line_item_quantity").get_attribute("value")
    self.assertEqual(qty_value, "3", "The item quantity is not the same we typed in!")
        
    cart_total = self.driver.find_element(By.CSS_SELECTOR, ".cart-total > .lead").text
    self.assertEqual(cart_total, "$68.97", "The total cart value does not match our expectations. That's a blocker!")

What’s new here?

If we look back at our first landing page test, you will notice that we use a lot of new things. We will cover them one by one now.

CSS selectors

One thing that catches the eye is that we looked for elements by using CSS expressions. You may have heard about them from your frontend peers, or maybe you have even worked with them yourself already. If not, don’t worry. We will cover them in a followup post. For now, I will give you a few translations:

[href='/t/spree']

„Give me a page element, whose attribute href is equal to ‚/t/spree'“. Quick reminder: href is used in link elements for defining the target URL that is opened, when the user clicks the link.

a[href*='spree-bag']

„Give me any link element, whose attribute href contains ’spree-bag‘.“

img[src$='/spree_bag.jpeg']

„Give me an image element, whose attribute src ends with ‚/spree_bag.jpg‘.“

input#quantity

„Give me an input element, whose HTML id is equal to ‚quantity‘.“ Yes, we could have used By.ID with „quantity“ here, but I wanted to be explicit about the element type input.

.line_item_quantity

„Give me an element that has line_item_quantity in its class attribute.“ The leading dot indicates that we are looking for elements that has the specified class. It is roughly equivalent to [class*='line_item_quantity'].

.cart-total > .lead

„Give me any element that has the lead class and that’s preceded by any element that has the cart-total class.“

By using CSS selectors you can be very specific about what element you want to use in your test case. This makes CSS selectors incredibly powerful and versatile while maintaining a solid degree of simplicity, especially compared to XPath (Ugh).

New properties and methods used in our test case

Aside from CSS selectors, we used several new methods and properties in our new automated test.

click() issues a click on the target element. Straight-forward.

.text contains the element’s written text. In <p>Some text</p> for example, we would get access to „Some text“.

get_attribute("someHtmlAttribute") gives us the content of an element’s HTML attribute. In our test we fetch an element’s value attribute that is often used to set the content of an input element. Another example might be the link element we talked about earlier. Given we want to have its href attribute for some further verifications. On our page:

<a id="mylink" href="https://www.example.org">an example link</a>

Then in our test we could do:

target_href = self.driver.find_element(By.ID,"mylink")
              .get_attribute("href")
self.do_some_shenanigans_on(target_href)

send_keys("My desired input") imitates key strokes on an element. We use it to type any string we want into our target text input element. In our test, that would be the „quantity“ input box.

And finally, with clear() we have a little helper that clears an input element from any content. In our test we use that to remove the preset value. Otherwise we would accidentally order 31 bags: „3“ from us and „1“ from the value preset by the page.

Continuing from here

Alright, let’s sum up what we accomplished:

We have seen how to open and close a browser window, do various actions on the page, verify their outcomes and finally we used that to write a fairly large cross test. And all of that on a full fledged ecommerce web app. Great job! Now where should we go next? For the next post, as promised, I’d like to go back one step and give you a deeper introduction to CSS expressions, because they will accompany you for a long time during your test automation journey. Afterwards we have to tackle a big problem you probably already noticed:

Copious amounts of repetition.

To tackle that we will leverage Behave, a Behavior Driven Development framework that makes it possible to write test cases in human-readable text form on top of a DRY Python code base. Afterwards we will apply the Page Object Pattern, a test automation – specific design pattern to reduce repetition even more.

Okay. To keep you busy while I am busy, how about finishing the checkout? As a reminder, you can find the full source code in my bitbucket repo. Or you can spoil yourself a little about Cucumber in Rust.

See you in the next post!

Home » Testautomation
Share it with:

AccessDenied in psutil

psutil is an interesting Python package that provides us with valuable insights about running processes, their memory and CPU usage and many more key aspects for monitoring and profiling processes across all major platforms. Thus, it is an incredibly useful tool for system admins, developers and testers alike. Now one of my favourite methods is cmdline(). Defined in the package’s Process class, it yields the whole CLI command of a running process in array form. There is one important catch on Windows though that got me kinda by surprise: While iterating casually through all my processes I suddenly got an AccessDenied error.

What did I try?

For an advanced verification in a process management – related test case I wanted to print the executed CLI command in a log file, so basically this:

import psutil

for pc in psutil.process_iter():
    print(pc.cmdline())

But that will just throw a painful AccessDenied error at us. Uff.

Why did that happen?

The reason is that psutil is quite consequent: It really lists every running process. That means even SYSTEM or root processes . That’s okay, it might even be interesting in one case or another, and you can still access selected attributes like the name() of a root process, if you want. But for me, it doesn’t suffice. I want to see the full-fledged cmdline(), but I understand that SYSTEM processes are none of my business. Once I came to term with that fact, the solution was easy: We just skip them.

The solution

What I did was applying a try-except around the loop’s inner statement:

import psutil

for pc in psutil.process_iter():
    try:
        print(pc.cmdline())
    except psutil.AccessDenied:
        continue

The continue statement will make sure that the processes that I’m not supposed to see are happily skipped without hurting the rest of the program flow.

But what if I need to monitor foreign processes?

In that case, we would need to execute the script within the process owner’s user context. That might be a bit fiddly in Windows depending on the use case, but of course that’s still possible. Just remember to keep the try-except block, because there still will be processes you wouldn’t be allowed to see.

Mine, for example.

Conclusion

So far for today. I hope this little Q3A (quick question quick answer) could shed some light upon that surprising AccessDenied error. If you want to learn more about psutil, I strongly recommend the readthedocs page. Otherwise, if you want to see more quick tips, I have one more for Python about environment variables. That one covers Python’s environment variable handling. As an alternative, if you are – like me – into containers, here is a handy docker ps trick useful for monitoring tasks as well.

Happy coding!

Home » Testautomation
Share it with:

Python Environment Variables: getenv() vs. environ[ ]

Last week, I was about to execute a test run using one of our functional test suites that requires a certain environment variable – and I forgot to set it, whoops.. The answer was harsh and generic: A good ol‘ Python KeyError. That put me up with two questions:

  1. What options do I have to get environment variables with Python and
  2. which one is the best?

Let’s start checking them out.

Introducing os.environ[key]

The first option is a simple dictionary that is prefilled with all environment variables when starting the Python process. This provides us with a simple-to-use interface that we already know and love.

Let’s try it out:

For python environment variables with os.environ[key], do the following: export MY_VAR="test", python3 -i, import os and os.environ["MY_VAR"]. This should yield 'test'.
Python environment variables with os.environ[key]

But what happens, when we query a variable that is not set? Then we hopefully prepared our try-except Block, because that call is going down the river faster than sound. This happens because dictionaries throw generic KeyErrors, when the queried key could not be found in the dict. That’s what happened in my test run.

os.getenv(key, default=None)

The second option follows a more higher-level approach by providing a function that takes our key in question and an optional default value. The usage is straight forward and it won’t crash as harshly as the first option does.

Let’s try it out:

For python environment variables with os.getenv(key, default), do the following: export MY_VAR="test", python3 -i, import os and os.getenv("MY_VAR"). This should yield 'test'. If you query MY_VAR2 instead, it should yield None. If you do os.getenv("MY_VAR2", "default"), it should yield default.
Python environment variables with os.getenv and default value handling

As we can see, we can now use any key we want; worst case is a result of None. But the increased flexibility comes with a price: We have to take care of whatever the function returns. Even harder, due to the fact that the default defaults to None (pun semi-intended), this solution is prone to hidden bugs. Therefore use it with care and set an appropriate default value if possible.

Conclusion: Which one works best for you?

We saw both options now, but which one works best (for you)? At the end of the day, it comes down to preference regarding two factors:

  1. Do you like your code to be more low-level or high-level?
  2. Do you want your fails fast or more controlled?

This is a decision you have to make, but once you have it, you find everything you need to get your environment variables with Python right here.

So long for the quick journey into my Python life. Usually I talk more about Rust as in my Cucumber Rust – tutorial and in this cheat sheet about Rust modules. But since Python is my main language on the job, there will definitely be more coming up soon. So if you are a friend of british comedy, feel free to stay tuned. Additionally, since container technologies are getting more and more important in my day job as well, I will write more about these techs, too, starting with this little docker ps – trick. Or, if you rather keep coding, check out my new Java threads tutorial.

Happy holidays and a merry Christmas Eeve!

Home » Testautomation
Share it with:

Cucumber-rust since 0.7 – The Most Important Changes

cucumber-rust has had a long way, since my last post about the 0.7 release in October 2020. It’s time to come back and see what happened since back then. First of all, starting from the initial 0.8.0 release, I will dig through the changelog and evaluate my favorite changes. Then we will update the Cucumber tests of my encrspyter project to the most recent version. Lots of stuff to do, so let’s go!

New things first

Let’s start soft: With 0.8.4, we got a --debug command line flag that leverages the test execution to nicely print stdout and stderr for each executed step. We can activate the debug mode in the runner creation code of our test’s main function:

fn main() {
    let runner = cucumber::Cucumber::<EncrsypterTestWorld>::new()
        .features(&["./tests/features/"])
        .steps(encrypt_decrypt_steps::steps())
        .debug(true); // This activates the new debug mode 
    ...
}

By running cargo test, we can see it in action:

Cucumber-rust's Debug mode produces sections in the test's cli output called Captured stdout and Captured stderr respectively. Captured stdout contains stdout text in white, Captured stderr contains stderr text in blue.

Neat, right?

t!-Macro extended with a World parameter type

Tiny but neat addition: We can now add the type of our Cukes World-object to the t!-closure.

t!(|mut world: MyPersonalCukesWorld, ctx| { [...] }

Although the generated code is the same as without the explicit type, it adds a bit more Rust-style expressivity. Sweet!

New callback methods for the Cucumber runner: before and after

In vanilla Cucumber, I admired its feature to define hooks that intercept the execution of a feature or a scenario. You can write some code and tell Cucumber to execute it before, after or before and after a scenario, feature or even a step. This is useful to for example set up or tear down a test database before or respectively after a test run.

With the release of 0.9.0, we can do similar things in Rust, too. There is a significant implementation difference to vanilla Cukes though: Our hooks won’t be picked up from wherever they are defined, but are defined as properties of the Cucumber runner instead. To compensate, our before and after hooks come with powerful query options to decide where to execute the defined method.

The second difference is that they are not officially called „hooks“ but „lifecycle methods“ instead. I might get this wrong due to habits. Please bear with me.

Lets head into an example. Given 2 features, one of them in English, one of them in German, each in 2 separate files:

# Feature 1 (English description)
Feature: Encrypt messages and write them to a file.

  Scenario: Encrypt a simple Hello World - message.
    Given I have an encryptor initialized with input "Hello World!"
     When I test print to STDOUT
      And I test print to STDERR
     Then I should see "Hello World!" in the test encryptor's input field
     When I encrypt the encryptor's input
     Then testfile.txt exists
      And testfile.txt is not empty
     When I decrypt testfile.txt
     Then the decrypted result should be "Hello World!"
# language: de
# Feature 1 (German description)
Funktionalität: Verschlüssele Nachrichten und schreibe sie in eine Datei.

  Beispiel: Encrypt a simple Hello World - message.
    Angenommen I have an encryptor initialized with input "Hello World!"
     Wenn I test print to STDOUT
      Und I test print to STDERR
     Dann I should see "Hello World!" in the test encryptor's input field
     Wenn I encrypt the encryptor's input
     Dann testfile.txt exists
      Und testfile.txt is not empty
     Wenn I decrypt testfile.txt
     Dann the decrypted result should be "Hello World!"

What we want to do now is get greeted and dismissed in the respective language. We will define proper lifecycle methods on our Cucumber runner to do that. In the main method:

    let english_feature_name = "Encrypt messages and write them to a file."; // full string filter for the English...
    let german_feature_pattern = Regex::new("Verschlüssele Nachrichten.*").unwrap(); // and a Regex filter for the German variant.

let runner = cucumber::Cucumber::<world::EncrsypterTestWorld>::new()
.features(&["./tests/features/"])
        .steps(crate::encrypt_decrypt_steps::steps())
        .language("de") 
        .before(feature(english_feature_name), |_ctx| {
            async { println!("Greetings, encryptor!") }.boxed()
        })
        .after(feature(english_feature_name), |_ctx| {
            async { println!("Goodbye, encryptor!") }.boxed()
        })
        .before(feature(german_feature_pattern.clone()), |_ctx| { // clone is necessary here due to the trait bounds of Inner<Pattern>
            async { println!("Hallo, Verschlüsselnder.") }.boxed()
        })
        .after(feature(german_feature_pattern), |_ctx| {
            async { println!("Tschüss, Verschlüsselnder.") }.boxed()
        });

feature() expects either the full feature description as a &str or a valid regex::Regex() matching your targets‘ description string. The latter requires the regex module as a dependency in your Cargo.toml, but it will provide you a highly powerful filtering tool, so adding that additional dependency is highly recommended.

Executing cargo test will show us what we expect. For the English feature file:

Greetings, encryptor!
Feature: Encrypt messages and write them to a file.

[...]

  ✔ Then the decrypted result should be "Hello World!"                                                                
Goodbye, encryptor!

For the German Feature file:

Hallo, Verschlüsselnder.
Funktionalität: Verschlüssele Nachrichten und schreibe sie in eine Datei.

[...]

  ✔ Dann the decrypted result should be "Hello World!"                                                               
Tschüss, Verschlüsselnder.

Great stuff! Last but not least, let me note that this does not only work with Feature, but with Scenario and Rule, too. You can even create more custom filters by combining them with And and Or. Please refer to the cucumber-rust code base for more about that.

Heads up, a breaking change!

With 0.9.0 we got one significant change in Cukes‘ public API, but don’t worry: Fixing it is quickly done and even quite easily automatable. If you review my guide on cucumber-rust for 0.7, you will see the related step definitions written like this:

.given_regex_async(
    r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#,
    t!(|mut world, texts_to_encrypt, _step| {
        world.encryptor.input = Cow::Owned(texts_to_encrypt[1].to_owned());
        world
    }),
)

This throws a compiler error now stating that the „signature“ of the t! macro has changed: Instead of the regex matches object in parameter #2 and _step in parameter #3, we now have a single StepContext object that contains the properties matches and step.

Therefore, in the above example we have to do the following:

  1. Remove the _step parameter entirely
  2. Rename our matches parameter texts_to_encrypt to something that reflects the StepContext type: ctx
  3. Replace the occurrences of texts_to_encrypt with ctx.matches[index_used_previously]

For _step we have no replacements to do, because we didn’t use it in the first place, so that’s basically it. The runnable step definition should now look like this:

.given_regex_async(r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#, t!(|mut world, ctx| {
                world.encryptor.input = Cow::Owned(ctx.matches[1].to_owned());
                world
    }),
)

Personally I like this particular change quite a lot, because it keeps the already loaded t! macro clean and organised. What do you think? Feel free to let me know in the comments below.

Feature: Add before and after lifecycle functions to the Cucumber builder. This function takes a selector for determining when to run 'before' or 'after', and a callback

Feature: add language argument to Cucumber builder to set default language for all feature files (ON HOLD)

Encrsypter’s Cucumber tests in a new look

I updated the tests in Encrsypter’s project master and in the cukes_0.9.0 branch, so if you want to see the changes in full action, give it a git pull on the master or a git checkout on the mentioned branch and enjoy.

Conclusion: great changes and improvements

Phew, so long. cucumber-rust really does have a long way, and many things have changed for more Cukes excitement. Personally I like the current implementation state really a lot and I’m looking forward to seeing its bright future. But for now, let’s wrap up the wrapup, shall we?

If you want to read more about Cukes in Rust, here’s my intro to Cucumber in Rust written for 0.7. Or you might say „meh, I prefer the vintage things of life, give me the vanilla stuff“. In that case, here you can find the original version of my intro guide.
And last but for sure not least, here’s the project’s full changelog with all the goodness listed. Happy cuking!

Home » Testautomation
Share it with:

Cucumber in Rust 0.7 – Beginner’s Tutorial

Introduction

Recently I have introduced us to Cucumber and how to use it in Rust, and while doing the writeup, cucumber-rust 0.7 has been released bringing a huge set of new and unique features. After a closer look through the readme, the strong focus on asynchronous test execution caught my eye. And since I’m a huge fan of ansynchronous programming having done lots of pet stuffs in NodeJS, seeing both my favorite BDD framework and my favorite system level language going strong in async got me severely hyped.

So let’s go!

Reminder: What is Cucumber?

Cucumber is a framework that implements Behavior Driven Development. The rules of BDD can be summarized as formulizing the requirements step by step in a more and more technical way. We start with the written requirements by your fellow business department and reformulate the requirements into a machine-readable format. Next, we use this text version to write an automated test case that fails, and implement the feature until the test passes. This flow gives it the popular resemblance to Test Driven Development. Cucumber leverages BDD by providing the machine- and human-readable layer based on so-called feature files. These use the Gherkin syntax, a simple syntax based on the keywords Given, When, Then, And and But.

Cucumber is still widely used as a test runner, although BDD is rarely actually applied due to the all-time-popular time limitation in nowaday’s software projects. Another rather unfortunate similarity to TDD.

Reminder: What is Rust?

Rust is a fairly new and rising system level programming language that operates in the same markets as C++ and friends. Besides system-level performance, its main focus lies in builtin security and safety. Furthermore, due to its security and safety-heavy design, it is able to completely omit automated memory management. It just doesn’t need it while still guaranteeing memory safety.

All these points are topped off by an exceptional developer experience: The Rust toolchain brings its full-fledged API documentation and its popular text book right to your command line-operating finger tips, and even compiler errors are designed as tiny educational lessons.

Our Test Object: A Simple AES Encryption Tool

In my previous post, we talked about a small encryption tool with the unspeakable name „Encrsypter“, which was started, when I did my first baby steps in Rust. Today it will serve us once more as our example test object.

The tool is based on aes-gcm, an AES encryption library (or „crate“ in Rust terms) that got audited successfully by the nccgroup. The full source code is available in my bitbucket repo, but for training purposes, I recommend removing the tests/ directory, as we will incrementally build it up during this tutorial.

Writing Cucumber-based Tests

Before we add the sources for our test cases, let’s check the test object’s project layout. We will start with the following directories and files:

encrsypter’s project directory without Cucumber tests. Here you find Cargo.toml, Cargo.lock and the src directory. In src/ you find constants.rs, decryptor.rs, encryptor.rs, lib.rs and main.rs.
encrsypter’s project directory without tests

Before we can start coding the test, we must add a cargo-compatible test subproject structure. On your favorite command line, please create the following directories with these terminal commands (all directories relative to the project root):

mkdir tests
mkdir tests/features

We will create and store our feature file that specifies the test steps of our Cucumber test in the features/ subdirectory. The step implementation will later go directly to the tests/ directory alongside the central configuration that we will create now. As described in the official documentation, we create a file called cucumber.rs in tests/ with the following content:

mod encrypt_decrypt_steps;

use async_trait::async_trait;
use encrsypter_lib::{decryptor, encryptor};
use std::borrow::Cow;
use std::convert::Infallible;

pub struct EncrsypterTestWorld {
    encryptor: encryptor::Encryptor<'static>,
    decryptor: decryptor::Decryptor<'static>,
    encrypted_base64: String,
    decrypt_result: String,
}

#[async_trait(?Send)]
impl cucumber::World for EncrsypterTestWorld {
    type Error = Infallible;

    // Much more straightforward than the Default Trait before. :)
    async fn new() -> Result<Self, Infallible> {
        let key = &[1; 32];
        let nonce = &[3; 12];

        Ok(Self {
            encryptor: encryptor::Encryptor {
                input: Cow::Borrowed(""),
                key,
                nonce,
            },
            decryptor: decryptor::Decryptor {
                file_path: "./testfile.txt",
                key,
                nonce,
            },
            encrypted_base64: "".to_string(),
            decrypt_result: "".to_string(),
        })
    }
}

fn main() {
    // Do any setup you need to do before running the Cucumber runner.
    // e.g. setup_some_db_thing()?;
    let runner = cucumber::Cucumber::<EncrsypterTestWorld>::new()
        .features(&["./tests/features/"])
        .steps(encrypt_decrypt_steps::steps());

    // You may choose any executor you like (Tokio, async-std, etc)
    // You may even have an async main, it doesn't matter. The point is that
    // Cucumber is composable. :)
    futures::executor::block_on(runner.run());
}

The EncrsypterTestWorld struct contains the mutable instances of our test objects: the encryptor and decryptor that serve to encrypt and decrypt our messages using AES. Further we will maintain special fields to keep track of the test object’s respective outputs. In version 0.7 we have an actual main function that serves as our entry point instead of the cucumber! macro in the previous version. Here we perform the basic configuration that gets our Cucumber test up and running: We…

  • … specify the test’s World struct containing our test objects, …
  • … tell Cucumber where to find feature files, …
  • … declare the module that contains our step implementations and …
  • … declare, which asynchronous executor we use to resolve the async step calls.

During this tutorial we use async-std supported by the futures and async-trait package. The latter is necessary to extend traits with asynchronous functionality that is not officially supported as of now (Rust 1.47.0). async-std is by no means set in stone though; you can use tokio or any other asynchronous runner equally well. I’m just much more familiar with async-std and futures.

The next config part is done in the project’s Cargo.toml. Again according to the official documentation, we should specify the dev-dependencies and the [[test]] directive as shown here:

[package]
name = "encrsypter"
version = "0.1.0"
authors = ["Florian Reinhard <me@florianreinhard.de>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
aes-gcm = "0.6.0"
rand = "0.7.3"

[lib]
name = "encrsypter_lib"
path = "src/lib.rs"

[[test]]
name = "cucumber"
harness = false # Allows Cucumber to print output instead of libtest

[dev-dependencies]
cucumber = { package = "cucumber_rust", version = "^0.7.0" }
base64 = "0.12.3"
futures = "0.3.6"
async-trait = "0.1.41"

In terms of dependencies we need the cucumber_rust package to run our tests and the futures and async-trait packages as discussed above.

Then we need the base64 package, because we will work with and do assertions on raw bytes. Although not entirely necessary, it may come in handy for visualisation purposes.

Under [[test]] we give our Cucumber test a name and we route the execution output to stdout to have a nice and tidy output, where we need it.

Alright, the config is done. Now we are ready to specify our first test case. We will encrypt a small „Hello World!“ message, give it a rough sanity check, and then we decrypt it back and hope that the decrypted output matches our input. Under ./tests/features, please create the file encryptor.feature. The containing test specification should roughly look like this:

Feature: Encrypt messages and write them to a file.

  Scenario: Encrypt a simple Hello World - message.
    Given I have an encryptor initialized with input "Hello World!"
     Then I should see "Hello World!" in the encryptor's input field
     When I encrypt the encryptor's input
     Then testfile.txt exists
      And testfile.txt is not empty
     When I decrypt testfile.txt
     Then the decrypted result should be "Hello World!"

This describes, what we want to accomplish: We want to encrypt the string „Hello World!“ and check, whether the output is there and whether it is not completely broken. Then we want to decrypt that output back and check, whether the output is the same as our input message. Next, we have to actually automate this test by implementing the Givens, Whens, Thens and Ands in the feature file.

Step Implementation Files

So far we have told Cucumber, where to find its stuff, and we created a written test specification. Great, we are almost there. The last step is to weave the magic into the Gherkin steps that do the heavy lifting, when Cucumber reads a step in the current feature file. Lets check out the following example step and see, what that means:

.when_async(
    "I encrypt the encryptor's input",
    t!(|world, _step| {
        world.encryptor.write_encrypted();
        world
    }),
)

This means whenever the Cucumber engine finds a step that matches „When I encrypt the encryptor’s input“ inside the feature file, the code within the closure that is constructed by the builtin t! macro is executed. Here we encrypt some random text.

The t! macro creates a wrapper around the step-implementing closure that extends it with asynchronous and future-driven functionality. It is exclusive to the asnychronous step methods. In the regular non-asynchronous step methods you can use regular closures.

Back to step implementations; regular expressions are usable, too:

.given_regex_async(
    r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#,
    t!(|mut world, texts_to_encrypt, _step| {
        world.encryptor.input = Cow::Owned(texts_to_encrypt[1].to_owned());
        world
    }),
)

This step defines the text that we want to encrypt using the When step from above. Here the text is derived from the feature file by matching the regular expression and its enclosing capture group ([\w\s!]+). The value that was read by the capture group goes to the custom closure parameter after world, in this case called text_to_encrypt. By using the regular expression above, we could have written the steps in our feature file like the following:

Given I have an encryptor initialized with input "Hi I am Floh"
=> encryptor input is "Hi I am Floh"

Given I have an encryptor initialized with input "99 bottles of beer on the wall…"
=> encryptor input is "99 bottles of beer on the wall…"

Given I have an encryptor initialized with input "Your ad here"
=> encryptor input is "Your ad here

Putting all the knowledge together, here is the sample implementation for our test steps. Please put it into ./tests/encrypt_decrypt_steps.rs (relative to the project root).

use cucumber::{t, Steps};
use std::borrow::Cow;
use std::fs;
use std::path::Path;

pub fn steps() -> Steps<crate::EncrsypterTestWorld> {
    let mut builder: Steps<crate::EncrsypterTestWorld> = Steps::new();

    builder
        .given_regex_async(
            r#"^I have an encryptor initialized with input "([\w\s!]+)"$"#,
            t!(|mut world, texts_to_encrypt, _step| {
                world.encryptor.input = Cow::Owned(texts_to_encrypt[1].to_owned());
                world
            }),
        )
        .then_regex_async(
            r#"^I should see "([\w\s!]+)" in the encryptor's input field$"#,
            t!(|world, expected_texts, _step| {
                assert_eq!(expected_texts[1], world.encryptor.input);
                world
            }),
        )
        .when_async(
            "I encrypt the encryptor's input",
            t!(|world, _step| {
                world.encryptor.write_encrypted();
                world
            }),
        )
        .then_async(
            "testfile.txt exists",
            t!(|_world, _step| {
                let testfile_path = Path::new("./testfile.txt");
                assert_eq!(testfile_path.exists(), true);
                _world
            }),
        )
        .then_async(
            "testfile.txt is not empty",
            t!(|mut world, _step| {
                let enc_message = fs::read("./testfile.txt").expect("Could not read test file.");
                world.encrypted_base64 = base64::encode(&enc_message);

                assert_eq!(world.encrypted_base64.len() > (0 as usize), true);
                world
            }),
        )
        .when_async(
            "I decrypt testfile.txt",
            t!(|mut world, _step| {
                world.decrypt_result = world.decryptor.read_decrypted();
                world
            }),
        )
        .then_regex_async(
            r#"^the decrypted result should be "([\w\s!]+)"$"#,
            t!(|mut world, expected_texts, _step| {
                assert_eq!(expected_texts[1], world.decrypt_result);
                world
            }),
        );

    builder
}

Please note that we use raw string literals written in r#...# in order to spare us escaping intentional doublequotes and backslashes.

Now we are ready for the first test run. Please execute the following command in your favorite terminal:

cargo test --test cucumber

If all goes well, it shows us a positive test result:

All 7 Cucumber feature steps passed. Yay!
All 7 Cucumber feature steps passed. Yay!

Conclusion: The All New Cucumber-Rust

The new version line cucumber-rust 0.7 brought a lot of super powers to the tips of our test automation fingers. With asynchronous tests, we are a huge step closer to real test parallelization and thus to less performance headaches, a quite notorious problem in test automation. The default trait got replaced by an intuitive and asynchronous World::new function, which makes working with Worlds much more intuitive, and as a great personal side effect, I got rid of the hassle that the World instance’s lifetime caused me. This helps me immensely to read, write and reason about the code. In future versions we might expect more simplifying changes to make asynchronous testing even more intuitive. For example with the power of procedural macros maybe we will get by without the t! macro ..?

I’m most certainly looking forward to the future versions.

If you are curious about how the test looked like in 0.6, here you can find my previous Cukes tutorial. Or if you’d like to know, here i talk about why I picked up test automation in the first place. And, as mentioned in my original Cucumber Rust article, here is my quick tutorial on how to use Rust Modules.

Have a great day & happy testing!

EDIT Nov. 2021: A lot of things have been worked on in Cucumber Rust, so I compiled a comprehensive summary about the most crucial changes in Cucumber Rust. Hope you enjoy it!

Home » Testautomation
Share it with:

Cucumber in Rust – Beginner’s Tutorial

Introduction

When I started my first QA role back in 2014, my first tasks included the maintenance and extension of a large test base, that was supposed to work for 4 different projects in parallel. It was based on Cucumber and the Ruby programming language, a stack I fell more and more in love with. This love still lasts to this day.

Therefore, it is time to relive the feeling, that is working with Cucumber from a fresh perspective, once more. To achieve this feeling we are going to apply an interesting little twist: We will code and test in the Rust programming language.

Rust, Ruby. 4 Letters and a capital R. Perfect!

What is Cucumber?

Cucumber is a framework, that implements Behavior Driven Development. The rules of BDD can be summarized as formulizing the requirements step by step in a more and more technical way. We start with the written requirements by your fellow business department and reformulate the requirements into a machine-readable format. Next, we use this text version to write an automated test case, that fails, and implement the feature up until the test passes. This flow gives it the popular resemblance to Test Driven Development. Cucumber leverages BDD by providing the machine- and human-readable layer based on so-called feature files. These use the Gherkin syntax, a simple syntax based on the keywords Given, When, Then, And and But.

Cucumber is still widely used as a test runner, although BDD is rarely actually implemented due to the all-time-popular time limitation in nowaday’s software projects. Another similarity to TDD, that is rather unfortunate.

What is Rust?

Rust is a fairly new and rising system level programming language, that operates in the same markets as C++ and friends. Besides system-level performance, its main focus lies in security and safety being builtin. Furthermore, due to its security and safety-heavy design architecture, it is able to completely omit automated memory management. It just doesn’t need it, while still guaranteeing memory safety.

All of these points are topped off by an exceptional developer experience: The Rust toolchain brings its full-fledged API documentation and its popular text book right to your command line-operating finger tips, and even compiler errors are designed as tiny educational lessons.

Our Test Object: A Simple AES Encryption Tool

I coded my first working Rust app, when I was learning its renowned ownership and borrow model. Usually, when it comes to the First App ™, I tend to write Fibonacci calculators in all kinds of setups: Fibonacci REST APIs, Fibonacci CLI calculators, Fibonacci FFI libs inside a Flutter app… But this time, i wanted something different. Something, that actually does stuff on a level worthy to let it be called a „system application“. So I decided to write a simple AES string encryption tool, that I gave the unspeakably cute name „Encrsypter“. It is based on aes-gcm, an AES encryption library (or „crate“ in Rust terms), that got audited successfully by the nccgroup a few months ago.

The full source code is available in my bitbucket repo. [Update: I’m currently working on an updated post with the new Cucumber-rs version. For this tutorial, please checkout the branch cukes_0.6.0 and, of course, stay tuned for the update.]

For training purposes, I recommend removing the tests/ directory, because we will successively build it up, as we go through the tutorial.

Writing Cucumber-based Tests

Before we add the sources for our test cases, let’s be aware of the test object’s project layout. We will start with the following directories and files:

Cargo.lock, Cargo.toml, src/, constants.rs, decryptor.rs, encryptor.rs and main.rs. No Cucumber tests yet.
encrsypter’s project directory without Cucumber tests

Before we can code the test, we must add a cargo-compatible subproject structure. On your favorite command line, please create the following directories with these terminal commands (all directories relative to the project root):

mkdir tests
mkdir tests/features
mkdir tests/steps

We will create and store our feature file, that specifies the test steps of our Cucumber test, in the features/ subdirectory, whereas the steps‘ implementations will go to steps/. But first of all, we will prepare the central configuration. As described in the official documentation, we create a file called cucumber.rs in tests/ with the following content:

#[path = "../src/encryptor.rs"] mod encryptor;
#[path = "../src/decryptor.rs"] mod decryptor;
#[path = "./steps/encrypt_decrypt_steps.rs"] mod encrypt_decrypt_steps;
use cucumber::cucumber;
use std::borrow::Cow;

pub struct World<'a> {
    encryptor: encryptor::Encryptor<'a>,
    decryptor: decryptor::Decryptor<'a>,
    encrypted_base64: String,
    decrypt_result: String
}

impl cucumber::World for World<'_> {}
impl std::default::Default for World<'_> {
    fn default() -> World<'static> {
        let key = &[1; 32];
        let nonce = &[3; 12];

        World { encryptor: encryptor::Encryptor{ input: Cow::Borrowed(""), key, nonce },
                decryptor: decryptor::Decryptor{ file_path: "./testfile.txt", key, nonce },
                encrypted_base64: "".to_string(),
                decrypt_result: "".to_string()
        }
    }
}

cucumber! {
    features: "./tests/features/", // Path to our feature files
    world: crate::World, // The world needs to be the same for steps and the main cucumber call
    steps: &[
        encrypt_decrypt_steps::steps // the `steps!` macro creates a `steps` function in a module
    ]
}

The World struct contains the mutable instances of our test objects: The encryptor and decryptor, that serve to encrypt and decrypt messages using AES. Further, we will maintain special fields to keep track of their respective outputs. The cucumber! block serves as our entry point, where we perform the basic configuration, that gets our Cucumber test up and running: We…

  • … tell Cucumber where to find feature files.
  • … specify the test’s World struct, that contains our test objects.
  • … declare the module, that contains our step implementations.

The next part of configuration is done in the project’s Cargo.toml. Again according to the official documentation, we should specify dependencies and a test directive like this:

[[test]]
name = "cucumber"
harness = false # Allows Cucumber to print output instead of libtest

[dev-dependencies]
cucumber = { package = "cucumber_rust", version = "^0.6.0" } 
base64 = "0.12.3"

In terms of dependencies, we need the cucumber_rust package to run our tests, then we need the base64 package, because we will work with and do assertions on raw bytes. Although not entirely necessary, it comes in handy for visualisation purposes.

Under [[test]], we give our Cucumber test a name, and we route execution outputs to stdout. We will see its use later, when we finally come to the executing part.

Alright, the config is done. Now we are ready to specify our first test. We will encrypt a small „Hello World!“ message, give it a rough sanity check, and then we decrypt it back and hope, that the decrypted output matches our input. Under ./tests/features, please create the file encryptor.feature. The containing test specification should roughly look like this:

Feature: Encrypt messages and write them to a file.

  Scenario: Encrypt a simple Hello World - message.
    Given I have an encryptor initialized with input "Hello World!"
     Then I should see "Hello World!" in the test encryptors input field
     When I encrypt the Encryptor's input
     Then testfile.txt exists
      And testfile.txt is not empty
     When I decrypt testfile.txt
     Then the decrypted result should be "Hello World!"

This describes, what we want to accomplish; we want to encrypt the string „Hello World!“, check, whether the output is there and whether it is not completely broken. Then we want to decrypt that output back and check, whether the output is the same as our input message. Next, we have to actually automate this test by implementing the Givens, Whens, Thens and Ands.

Step Implementation Files

So far we have told Cucumber, where to find its stuff, and we created a written test specification. Great, we are almost there. The last step is to weave the magic into the Gherkin steps, that do the heavy lifting, when Cucumber reads a step in the current feature file. Lets check out the following example step and see, what that means:

when "I encrypt the Encryptor's input" |world, _step| {
    world.encryptor.write_encrypted();
};

This means whenever the Cucumber engine finds a step, that matches „When I encrypt the Encryptor’s input“ inside the feature file, the code within the closure is executed. Here, we encrypt some random text.

Regular expressions are usable, too:

given regex r#"^I have an encryptor initialized with input "([\w,\s,!]+)"$"# (String) |world, text_to_encrypt, _step| {
        // the # are necessary to prevent the inner quotations marks as part of the String
        world.encryptor.input = Cow::Owned(text_to_encrypt);
    };

This step defines the text, that we want to encrypt using the When step from above. Here, the text is derived from the feature file by matching the regular expression in r# and the enclosing capture group ([\w,\s,!]+). The value, that was read by the capture group, goes to the closure parameter after world, in this case text_to_encrypt. Note that the „r“ in r# stands for „raw string“ instead of „regular expression“. Raw strings are a means to spare us from copious amounts of escape slashes within the regular expression string; otherwise, they are regular strings. I won’t go into too much detail here. If you want to learn more about them, check out this post about raw string literals.

By using the regular expression above, we could have written the steps in our feature file like the following:

Given I have an encryptor initialized with input "Hi I am Floh"
=> encryptor input is "Hi I am Floh"

Given I have an encryptor initialized with input "99 bottles of beer on the wall…"
=> encryptor input is "99 bottles of beer on the wall…"

Given I have an encryptor initialized with input "Your ad here"
=> encryptor input is "Your ad here"

Putting all the knowledge together, here is the sample implementation for our test steps. Please put it into ./tests/steps/encrypt_decrypt_steps.rs (related to the project root).

use cucumber::steps;
use std::fs;
use std::path::Path;
use std::borrow::Cow;

steps!(crate::World<'static> => {
    given regex r#"^I have an encryptor initialized with input "([\w,\s,!]+)"$"# (String) |world, text_to_encrypt, _step| {
        // the # are necessary to prevent the inner quotations marks as part of the String
        world.encryptor.input = Cow::Owned(text_to_encrypt);
    };

    then regex r#"^I should see "([\w,\s,!]+)" in the test encryptors input field"# (String) |world, expected_text, _step| {
        assert_eq!(expected_text, world.encryptor.input);
    };

    when "I encrypt the Encryptor's input" |world, _step| {
        world.encryptor.write_encrypted();
    };

    then "testfile.txt exists" |_world, _step| {
       let testfile_path = Path::new("./testfile.txt");
       assert_eq!(testfile_path.exists(), true);
    };

    then "testfile.txt is not empty" |world, _step| {
        let enc_message = fs::read("./testfile.txt").expect("Could not read test file.");
        world.encrypted_base64 = base64::encode(&enc_message);

        assert_eq!(world.encrypted_base64.len() > (0 as usize), true);
    };

    when "I decrypt testfile.txt" |world, _step| {
        world.decrypt_result = world.decryptor.read_decrypted();
    };

    then regex r#"^the decrypted result should be "([\w,\s,!]+)"$"# (String) |world, expected_text, _step| {
        assert_eq!(expected_text, world.decrypt_result);
    };
});

Now we are ready for the first test run. Please execute the following in your favorite terminal:

cargo test --test cucumber

If all goes well, it shows us a positive test result:

Positive result of our Cucumber test. 1 fearture with 1 scenario containing 7 steps, all green and checkmarked in the terminal.
The test passed. Yay!

Conclusion: Cucumber in Different Languages

This is by no means the end of Cucumber’s options and possibilities. There are many many many more well maintained ports for many different platforms out there. Not all of them may be offcial, e.g. the Rust port we used today, but they are nonetheless maintained and fully functional. And they contributes to its well deserved popularity as well as the official ports. This is what counts in the end.

For Cucumber-rs it’s not the end of possibilities, too, as version 0.7 has been released recently. It brings asynchronous test support and a new builder-based approach to the table. I’m hyped to try it out, especially because I love asynchronous coding (Please don’t judge me..)

But for now, this is a good starting point to read more about other facettes of test automation. For example, you can learn how to set up a Zalenium cluster for distributed browser UI testing. It is well-suited to be combined with Cucumber. If you’d rather learn more about unconventional and unstructured automation testing, you might like my article about fuzzing in Java. Also, since I make heavy use of Rust Modules, I have written a quick tutorial about how to use them without hassle.

Have a great day!

EDIT Nov. 2021: A lot of things have been worked on in Cucumber Rust, so I compiled a comprehensive summary about the most crucial changes. Hope you enjoy it!

And btw, if you are interested in test automation, but Rust is not particularly your cup of tea: I’ve compiled an entry-level TA tutorial with Python. Check it out!

Home » Testautomation
Share it with:

Fuzzing in Java – How and Why

Back in Summer 2019 we had a workweek full of tech talks and presentations, where we explored various topics from advanced DevOps practises to biometric engines. We had eeeverything. Of course testing-me had to live up to his urge and enrolled to all listed talks regarding his favorite IT-discipline: System Design Processes, Enterprise-Scale QA… and then there was that particular presentation about a simple yet effective test automation technique called fuzzing.

That one got me. I listened with an evil grin and decided to give it a shot. And that’s what we are going to do today!

Fuzzing – as explained in the talk – is a testing technique, that feeds the application a huge amount of random input data with different types and checks, which of them crashes the application. Simple enough. This can happen in a Black Box fashion by bombarding the public API or in a more White Box fashion by instrumenting the application code in order to get even more coverage and insides-related details.

In today’s tutorial, we will go through a Black Box Fuzzing setup written in Java. That’s because I’m more of a Black Box Testing person, and my main field of action is Java-based enterprise applications. More exactly, we will prepare a happy little Play Framework-based web application, that somehow got a commercial 3rd party conversion library called „Legacy“ imposed upon. Next we QAs want to have a first glance at Legacy’s state of quality to see, whether the purchase was at least somewhat worth it.

Prerequisites

This tutorial assumes that you have sbt and Maven installed. Since I wanted to try the Play Framework as a nice little side learning, we have to get along with sbt, but don’t worry: We need it only to compile the app. If you are curious, you can use it to run the app, too, but that’s 100% optional.

Maven on the other hand is used to operate the fuzz tests and thus will be our bread and butter tool.

Our Setup

Here’s the link to Happy Little Webapp’s source code repository. In ./app you can find the code of our Legacy-Module next to the web app’s controllers and (unused) views. Technically it’s not a blackbox, since I had to write the example code by myself, but let’s assume, we as the testers don’t know anything about it’s details, except for the public methods‘ signatures.

First, open the sbt shell: In your terminal of choice, enter the command sbt. Next, in the sbt-shell we just opened, we enter compile to compile the app’s code. Afterwards, if you are curious about what the app actually does, you can type run to start it. Now you can perform a request in your browser like:

http://localhost:9000/dollar2euro/58

It should display 53.36. Not as correct as we would expect it to be, because the used factor for the calculation is static and likely outdated. But for testing purposes, let’s assume, it is sufficient.

Next, we take care of our fuzz tests located in ./fuzztests. The pom.xml already knows about their location, so by using it, we can execute the tests right away. The fuzzing will be executed with a maven plugin called jqf-fuzz. Please see its github repository for the code and its well-elaborated documentation. With all that coming together, we are ready to fuzz.

Get the fuzzing started

First, we have to install the jqf-fuzz Maven plugin by doing a simple:

mvn clean test-compile

This downloads the jqf-fuzz plugin to our local maven repository and compiles the test sources. Now we have access to 2 new maven goals: jqf:fuzz executes the fuzz tests, and jqf:repro replays failed test cases to hunt down associated defects. Both goals expect several input parameter defined by JVM parameters (-D on the CLI) and/or by definition within the POM. This allows for a rich set of customization, that is both user- and CI-friendly. For demonstration purposes, I already configured the parameter time in the POM so that the test runs for 10 seconds, that still provides us with lots of input. Further, I predefined the fuzz test class to be executed. Therefore, the only parameter we must provide from the terminal is our test method -Dmethod=dollar2euro. We will do that in a minute, but first let’s have a look at the fuzz test class.

Let’s run the test

This is what we gonna unleash upon our web app:

@RunWith(JQF.class)
public class LegacyConverterFuzzer {

private static LegacyConverter legacyConverter;

@BeforeClass
public static void beforeClass(){
    legacyConverter = new LegacyConverter();
}

@Fuzz
public void dollar2euro(Object input){ // this is where the fun things happen
   try {
       System.out.println("Input: " + input.toString());
       System.out.println("Output: " + legacyConverter.dollar2euro(input));
   } catch (Throwable e) {
       System.out.println(e.getClass().getName() + ":" + 
                          e.getMessage());
   }
}

[... some more Fuzz-Tests, please see the repository linked above...]
}

Legacy’s executives promised, that any input works fine. Okay! Then we perform the test dynamic-typed by using an Object-typed input parameter.

Alright, that’s the code. Let’s fire it up. On your terminal, please do:

mvn jqf:fuzz -Dmethod=dollar2euro

Here’s an excerpt from the results as seen in my terminal. The output will vary for each new test run, because, as we said earlier, the input values in fuzz tests are random.

java.lang.NumberFormatException: Character 텈 is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
Input: edu.berkeley.cs.jqf.fuzz.junit.quickcheck.InputStreamGenerator$1@4fc3c165
java.lang.NumberFormatException: Too many nonzero exponent digits.
Input: 뤇皽
java.lang.NumberFormatException: Character 뤇 is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
Input: ky
java.lang.NumberFormatException: Character k is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
Input: FixedClock[+898773291-08-05T17:23:55.165612278Z,UTC]
java.lang.NumberFormatException: Character array is missing "e" notation exponential mark.
Input: -8475850143961316955
Output: -7797782132444411598.60
Input: bn
java.lang.NumberFormatException: Character b is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
Input: 16:19:25.242056065Z
java.lang.NumberFormatException: Character array is missing "e" notation exponential mark.
Input: -895394919-05-23T23:50:04.780324820
java.lang.NumberFormatException: Character array is missing "e" notation exponential mark.
Input: 11:14:21.890848137Z

Phew! We got a lot of NumberFormatExceptions. So much about „any input works“. Our PO should know about that.

6 months full of arguments later, the supplier delivered API version v1.0.1 of his LegacyConverter ensuring a static-typed API. He changed dollar2euro to the following:

    public String dollar2euro(BigDecimal input){
        BigDecimal dollars = input.setScale(2, BigDecimal.ROUND_HALF_EVEN);
[...]
}

Of course, we have to adapt our controller, too. For playground reasons, we keep that change as simple as possible.

    public Result dollar2euro(String dollars) { 
        return ok(importantConverter.dollar2euro(
                  BigDecimal.valueOf(Double.valueOf(dollars))));
    }

When we enter non-numeric inputs, the app will still fail, but at least it’s on us now.

Alright, the fixes are applied. Now in our test class, we see a sweet little type check error: We have to change the test method’s input parameter’s type accordingly to BigDecimal, too. This makes our fuzz test static-typed.

Afterwards, we recompile the tests and repeat the fuzz:

mvn clean test-compile
mvn jqf:fuzz -Dmethod=dollar2euro

giving us (excerpt):

Input: 152
Output: 139.84
Input: -1000
Output: -920.00
Input: -771298122
Output: -709594272.24
Input: 80372941329620235
Output: 73943106023250616.20
Input: 272536
Output: 250733.12
Input: -1000
Output: -920.00
Input: -2625164447481769740006272317
Output: -2415151291683228160805770531.64
Input: 9340202544
Output: 8592986340.48
Input: -34567
Output: -31801.64
Input: 17223398969630190416957297
Output: 15845527052059775183600713.24

Much better!

Conclusion – What did we achieve by fuzzing?

We have seen, how we can use fuzzing to create a vast storm of static or dynamic-typed test inputs and thus create hundreds of different test cases. From the output logs we can learn, what inputs can be handled by our application and – more interesting – what not. This provides us with an insightful first glance at the quality, a great starting point for further functional test cases, and, of course, with even more application bombing by using our favorite CI system.

From here, we can follow the functional testing track with even more elaborated automation or dive deeper into Java Fuzzing with the official JQF-Fuzz paper. And if you still need motivation to automate your tests, check out here why test automation is cool. Also there is a new tutorial about threads in Java fresh out of the oven. Enjoy it while it’s hot!

Last but not least a huge shoutout to the great people at X41 D-SEC, who held the exciting talk that inspired me and made me put fuzzing into my tool box.

Home » Testautomation
Share it with:

Playwright for Browser Automation

Last week I held a short & sweet presentation in the company about the usage, benefits and drawbacks of Browser Remote Debugging APIs. One unfortunate problem we discovered was the lack of a standard across the browsers; every major browser maintains its very own implementation. The RemoteDebug – Intitiative tried to solve this problem, but until now without noticeable success, as you can see here by the lack of activity. Therefore, the Test and Development – World needed to deal with that all by themselves. A great team of ex Puppeteer-developers, who moved from Google to Microsoft, did exactly that by bringing us Playwright, a framework for writing automated tests encapsulating and using the various Remote Debugging Interfaces. In today’s short example we write a quick example test with Playwright.

Installing Playwright

As a starting prerequisite, we need a NodeJS-Distribution with Version 10 or greater. Next, we go to our already well-filled project directory and create a new NodeJS-Project:

$ cd /path/to/your/project/directory
$ mkdir playwright_test && cd playwright_test
$ npm init
$ npm install --save-dev playwright

While the installation progresses, you will notice that Playwright brings its own browser binaries. Don’t worry about that, they are still perfectly valid, as the rendering engines are not modified at all. Only the debugging capabilities have been given a few extensions.

Alright, that’s all we need.

Time to dive into the code!

Let’s assume we want to buy red shoes on Amazon, because we need new shoes, and red is a nice color.

// 1. We start by initializing and launching a non-headless Firefox 
// for demo purposes.
// (How do you call them, "headful"? "headded"? Feel free to drop me 
// your best shots. :))
const {firefox} = require("playwright");

(async () => {
  const browser = await firefox.launch({headless: false, slowMo: 50});
  const context = await browser.newContext();

  // 2. Next, we head to the Amazon Landing Page...
  const page = await context.newPage();
  await page.goto("https://www.amazon.com");
  
  // 3. ...do the search for Red Shoes...
  await page.fill("#twotabsearchtextbox", "Red Shoes");
  const searchBox = await page.$("#twotabsearchtextbox");
  await searchBox.press("Enter");

  // 4. ...and take a nice deep look at the page 
  // by saving a screenshot.
  await page.waitFor("img[data-image-latency='s-product-image']");
  await page.screenshot({path: "./screenshot.jpg", type: "jpeg"});
  
  // 5. Afterwards, we leave the testrun with a clean state.
  await browser.close();
})();

That’s it for now. From here, we can extend the test by doing elaborate verification steps, check out a nice pair of red shoes and pay them with our hard-earned testing money.  Feel free to check out the example’s full source code from here and go ham.

Conclusion

With Playwright we got a means to write automated tests with ease against the many different Remote Debugging APIs. It copes nicely with the API differences while preserving an intuitive and familiar JS test automation syntax.

So if you are looking for a more lightweight and lower level alternative to Selenium, give it a go!

Home » Testautomation
Share it with: