Kategorie: CI

Posts about Continuous Integration

Veröffentlicht am

Problem reading geckodriver versions: error sending request for url

Today, while hunting down a nasty issue in a longer Selenium/Cukes scenario outline, I was flooded with a particular annoying warning:

WARNING:
org.openqa.selenium.manager.SeleniumManagerlambda$runCommand$1
Problem reading geckodriver versions: error sending request for
url (https://raw.githubusercontent.com/SeleniumHQ/selenium/trunk/common/geckodriver/geckodriversupport.json).
Using latest geckodriver version

org.openga.selenium.manager.SeleniumManagerlambda$runCommand$1
WARNING: Exception managing firefox: error sending request for
url (https://github.com/mozilla/geckodriver/releases/latest)

You too? Then you are at the right place.

The reason

Selenium is trying to check for an update for your geckodriver executable. What is planned as a nice convenience feature can be quite an annoyance in a restricted environment. When Selenium is executed within such an environment, it cannot contact the remote geckodriver version dictionary and nags us with „Problem reading geckodriver versions: error sending request for url …“.

Thankfully the Selenium team seems to be aware of that and provided us with a simple fix. Sadly that fix is quite non-explicit but that’s not going to stop us anytime soon.

Fixing the „Problem reading geckodriver versions: error sending request for url“ warning

The update check happens if – and only if – Selenium derives the geckodriver path from Seleniums automatic lookup feature (i.e. when it checks the system PATH variable). Therefore all we have to do is setting the geckodriver path explicitly. In Java we do that like this:

System.setProperty(“webdriver.gecko.driver”, “/path/to/your/geckodriver”);

By setting that Java system property we tell Selenium explicitly what geckodriver to use and we prevent it from doing any additional shenanigans on the driver binary thus silencing the warning. Using an appropriate if-clause you can tailor it to any kind of environment. You have full internet access? Cool! Then your system geckodriver might be perfectly sufficient. It’s all in your hands.

Final thoughts

Alright! This should relieve you from that „Problem reading geckodriver versions: error sending request for url“ warning. When working with complex Selenium Webdriver tests, it is overall nice that the framework logs so verbose and provides us with all these features. But sometimes it can be a little full of itself. The good thing is though that there usually is a way to customise Selenium’s behavior to our personal needs, whether it is working in a restricted network area or with full access to the internet – Selenium knows it all.

Though sometimes it has its difficulties to articulate itself – just as we humans do. Anyways, thank you for reading. If you are still curious about what’s possible with Java in QA, here’s an interesting post about fuzzing in Java. If you don’t particularly care about Java but you would like to see more test automation action in general, I’d recommend trying Cucumber in Rust.

Have a great day!

Share it with:
Veröffentlicht am

Maven SSL and HTTPS Configuration – a Howto

In the world of Java development, working with Maven repositories is essential, as these repositories host the dependencies that our applications rely on. While connecting to public repositories like Maven Central is straightforward, accessing private or secured Maven repositories requires additional configuration. Specifically, when connecting over HTTPS, a Java Keystore may need to be configured to trust the repository’s SSL certificate. Here’s how to set up a Java Keystore for secure HTTPS connections to a Maven repository, ensuring a seamless and secure build process.

Why configure a keystore?

Java applications, including Maven, use the Java Keystore (JKS) to manage certificates for secure communication over SSL/TLS. If your Maven repository uses HTTPS with a self-signed or non-standard certificate, Maven might reject the connection because it cannot verify the certificate’s authenticity. By adding the repository’s certificate to a trusted keystore, you’re explicitly telling Java and Maven to trust this certificate. This setup is particularly relevant for organizations using internal repositories or third-party vendors with their own certificate infrastructure.

Create a test certificate chain

At first we build a custom certificate for test purposes. Please see the following few lines of bash:

# Step 1: Generate Root CA key and self-signed certificate
openssl genrsa -out rootCA.key 4096
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem -subj "/C=US/ST=State/L=City/O=MyCompany/OU=Org/CN=RootCA"

# Step 2: Generate End-User key and CSR
openssl genrsa -out endUser.key 4096
openssl req -new -key endUser.key -out endUser.csr -subj "/C=US/ST=State/L=City/O=MyCompany/OU=Org/CN=EndUser"

# Step 3: Sign the End-User CSR with the Root CA to create the end-user certificate
openssl x509 -req -in endUser.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out endUser.crt -days 500 -sha256

Create a Java Keystore (JKS) from certificate and key

Next we build the JKS file from our new certificate and key:

openssl pkcs12 \ 
    -export \ 
    -in endUser.crt \
    -inkey endUser.key \
    -out temp.p12 \
    -name personalcreds \
    -password pass:sometemporarypw      	
    
keytool \
    -importkeystore \
    -deststorepass YOUR_DESIRED_JKS_PW \
    -destkeystore somename.jks \
    -srckeystore temp.p12 \ 
    -srcstoretype PKCS12 \ 
    -srcstorepass sometemporarypw \
    -alias personalcreds

rm -f temp.p12

Set up and use Maven’s .mvnrc file

Configuring SSL for Maven using a .mavenrc file may look familiar to you as it involves setting up environment variables for your Java KeyStore (JKS) settings:

export MAVEN_OPTS="-Djavax.net.ssl.keyStore=/path/to/your/keystore.jks \
-Djavax.net.ssl.keyStorePassword=yourKeystorePassword \
-Djavax.net.ssl.trustStore=/path/to/your/truststore.jks \
-Djavax.net.ssl.trustStorePassword=yourTruststorePassword"

Put the file somewhere where your shell is able to consume it or run

$ source /path/to/your/.mavenrc

yourself to have your SSL config up and running.

Configure .mvn/jvm.config file for Maven SSL Authentication

A .mvn/jvm.config keeps your project well-structured with regards to separation of concern and allows for an easy Maven SSL configuration. Therefore this is the way I recommend. To use it properly create a .mvn directory in your Maven project’s root directory, put it on .gitignore (or an equivalent file depending on your VCS) and place a file jvm.config in there. Here we place our Maven SSL config like this:

-Djavax.net.ssl.keyStore=/path/to/your/keystore.jks
-Djavax.net.ssl.keyStorePassword=yourKeystorePassword
-Djavax.net.ssl.trustStore=/path/to/your/truststore.jks
-Djavax.net.ssl.trustStorePassword=yourTruststorePassword

Use the Maven settings.xml for SSL config

If you prefer to have the whole authentication process at one place there’s always the settings.xml for you:

<servers>
	<server>
    	<id>your-secured-repo</id>
    	<configuration>
        	<javax.net.ssl.keyStore>path/to/your/maven_truststore.jks</javax.net.ssl.keyStore>
        	<javax.net.ssl.keyStorePassword>changeit</javax.net.ssl.keyStorePassword>
        	<javax.net.ssl.trustStore>path/to/your/maven_truststore.jks</javax.net.ssl.trustStore>
        	<javax.net.ssl.trustStorePassword>changeit</javax.net.ssl.trustStorePassword>
    	</configuration>
	</server>
</servers>

Benefits of a custom keystore configuration

Setting up a custom keystore for Maven offers flexibility and security. It enables secure connections to internal repositories or those requiring special certificates, without affecting the global Java truststore. Additionally, it simplifies management when dealing with multiple repositories or environments, as each project or CI/CD pipeline can use its own keystore as needed.

Final Thoughts

While configuring a Java Keystore may seem daunting, it’s a powerful way to manage secure connections to Maven repositories. By following these steps, you can ensure that your Java applications are securely retrieving dependencies, protecting your codebase and development environment from potential threats. So next time you encounter an SSL error connecting to a repository, remember—it might just be a keystore configuration away from a quick fix! The full CA and JKS script can be found in this gist, and if you are not fed up with Java, here’s an interesting post covering Browser Test Automation in Java. If you’d rather see something else, don’t worry, here’s the Python version. One more I’d like to share with you: Fresh from the oven comes a post, where I explain the SQLException „before start of resultset“. Give it a click and and see you next time!

Share it with:
Veröffentlicht am

Gitlab CI Error – Cannot create local repository at: /.m2/repository

When you are using downstream pipelines in Gitlab CI for complex Maven Projects as I do, you may have stumbled accross this error at least once: Cannot create local repository at: /.m2/repository. The Gitlab runner tries to create a local Maven repository at /.m2/repository – so in the topmost directory – and fails horribly due to an obvious lack of permissions. Let‘ see what happens here.

The Gitlab CI Setup

In my case I had a shell-based Gitlab runner that executes a test suite based on a few artifacts published by a Docker runner. Due to versioning and system test level realism reasons I was not able to use the artifacts directly. Unfortunate, but it shouldn’t pose that kind of a problem, does it?

Now when it comes to cloning the test suite on my shell runner, this cryptic error has been dropped. And naturally I was like „WTF is this runner trying to do!?“

The root cause in Gitlab CI downstream jobs

What I was not aware of is that per default the „child job“ inherits all the custom and Gitlab-provided runner variables from the triggering parent job that is still executed on a Docker runner. Now since the trigger happens from a Docker runner job, the variables my child job receives are poison for a baremetal environment that is my shell runner. Not least because things actually do happen at / in a Docker-based Gitlab execution environment, which is perfectly fine, but not on a shell runner.

The solution: What I had to do to make the CI jobs work

To fix the problem I had to set this on my trigger job definition:

inherit:
  variables: false

This did not just fix the error for me, but it also made perfect sense. Due to the different execution environments – shell vs. docker and project A vs. project B – I have a different set of requirements for my Gitlab CI test job. Therefore we have another case of an error leading to better software design. In addition I learned another piece of Gitlab’s sometimes quite obscure default settings.

Conclusion

I hope this helps you during your day to day journey through the jungle that are Gitlab CI downstream jobs. As I’m an avid QA engineer, so if you want to read more about writing actual automated tests. Also I take care of deeper coding basics like working with threads in java. If you’d rather want to read up about Gitlab CI’s inherit keyword, here’s the link to the relevant section of the official Gitlab CI documentation. Feel free to have a look!

Have a great weekend everybody!

Share it with: