Monat: März 2025

Veröffentlicht am

Python Child Processes and subprocess run

If Python likes to do except for KI and prey on mice it is automation. Test automation, process automation, tiny little helper supporting finance people all over the world… automation is on everyone’s lips – and on everyone’s project schedules. And due to the nature of automation in fields other than regular software development we might want to use what the operating system is already providing us. That’s where subprocess run comes in handy to create child processes in no time. We will see how to start an external child process from within python and we will use it to implement a simple automation use case. Let’s-a-go.

Quick child process sample with subprocess run

Okay let’s not go that fast. Before we head into the real thing, let’s train with a simple example. We will let the OS print the today’s date through a Python child pocess. Please see the following listing:

import subprocess

process = subprocess.run(["date"], capture_output=True, encoding="UTF-8")
print(process.stdout)

Execute that code in your favorite console or IDE. You should get an output line similar to this:

Sun Mar 9 19:01:22 CET 2025

The first parameter is pretty straight forward. We provide the run method with the program command we would like to start using the list notation – date in this case. Another popular example could be ["ls", "-al"]. Try it out by replacing the date command with that one. In the meantime I tell you what the other method parameters do.

capture_output=True

This makes sure that we save the output in process.stdout for later use instead of just firing the command and forgetting the output. That would be kinda not-handy when we automate more complex things.

encoding=“UTF-8″

Without setting a nice string encoding we would get a byte-representation of process.stdout when printing that to the console. That’s not nice. Therefore we set the encoding to UTF-8 and get a nice and clean output line.

Advanced automation example: Create an X509 certificate using child processes

Alright, now that we a warmed up we will tackle a more realworld automation example. Let’s imagine you are tasked to secure a Maven repository with TLS and you need to generate an X509 certificate. If you are an avid reader of my blog this might sound familiar to you and you are right! Just recently I posted about SSL/TLS for Maven repos where we generate a Java Keystore and use that to connect to a repo via HTTPS. For this post here we take the certificate generation shell script and execute that from python. Let’s see how we achieve that:

import subprocess as sub

# Step 1: Generate Root CA key and self-signed certificate
p = sub.run(["openssl", "genrsa", "-out", "rootCA.key", "4096"],
            check=True, capture_output=True, encoding="UTF-8"
            )
print(p.stdout)

p = sub.run(["openssl", "req", "-x509", "-new", "-nodes", "-key",
             "rootCA.key", "-sha256", "-days", "1024", "-out", "rootCA.pem",
             "-subj", "/C=US/ST=State/L=City/O=MyCompany/OU=Org/CN=RootCA"],
            check=True, capture_output=True, encoding="UTF-8"
            )
print(p.stdout)

# Step 2: Generate End-User key and CSR
p = sub.run(["openssl", "genrsa", "-out", "endUser.key", "4096"],
            check=True, capture_output=True, encoding="UTF-8"
            )
print(p.stdout)

p = sub.run(["openssl", "req", "-new", "-key",
             "endUser.key", "-out", "endUser.csr",
             "-subj", "/C=US/ST=State/L=City/O=MyCompany/OU=Org/CN=EndUser"],
            check=True, capture_output=True, encoding="UTF-8"
            )
print(p.stdout)

# Step 3: Sign the End-User CSR with the Root CA to create the end-user certificate
p = sub.run(["openssl", "x509", "-req", "-in", "endUser.csr",
             "-CA", "rootCA.pem", "-CAkey", "rootCA.key", "-CAcreateserial",
             "-out", "endUser.crt", "-days", "500", "-sha256"],
            check=True, capture_output=True, encoding="UTF-8"
            )
print(p.stdout)

Execute that in a separate .py file in your favorite IDE and you should receive a neat set of X509 files including our target: endUser.crt.

My favorite method options for subprocess run

Now we got to know capture_output and encoding already. What else does subprocess run support? Here are 3 of my favorite method parameters:

check

If this is set to True and the child process terminates with a non-zero exit code – i.e. terminates with an error – a CalledProcessError is thrown. If False the process just terminates and your program will go on as if nothing happened. I am a big fan of failing fast so this is big sugar to me.

shell

If set to True your child process can use advanced shell features like pipes and filename wildcards. This is especially useful when it comes to various IT automation tasks.

env

This takes a str-to-str-dictionary containing your personal custom set of environment variables tailored to your child process by you. This set overrides the default set that is derived from its parent. This comes in handy if you want to start your process in a more isolated way.

You are curious and want more now? Feel free to check the Python subprocess docs. There you’ll be served.

Final thoughts

Phew! Now we definitely earned our after-work soda. We have seen what the subprocess run – method is capable of and we solved real-world problems using child processes provided by that. We got ready to automate aaall the things that’s for sure. Wait, you still want more automation with Python? If you really do and you are willing to dip into software testing I have one to offer that covers test automation in Python. Alas it’s automation, too. Right? Or did you think that Python is not your thing now but you got inspired and want more shell? That’s perfectly fine, too. Here we create Linux users directly in your favorite shell.

Share it with:
Veröffentlicht am

Docker Compose: Custom Commands to Keep your Container up

Sometimes you just want to keep up your container without running the intended process, for example to diagnose an unexpected shutdown during starting up the container’s process. To achieve that, we make use of the option in docker and docker-compose to overwrite the container’s innate command that is usually executed during your container’s runtime.

Preparing our experiment

First of all we gotta pull the docker image for our experiments. For illustration purposes we use an image that is completely unrelated to any command we’re going to use later:

docker pull maven:3.9.9-eclipse-temurin-8-alpine

That’s all. Now let’s head into our experiments.

docker container run COMMAND – parameter

When executing docker run we can not only set an image name but also add a custom command that overrides the image’s default command. Let me illustrate that by showing you the date of today as of writing this post – using maven’s docker image.

>docker run maven:3.9.9-eclipse-temurin-8-alpine date
Sat Mar 8 14:21:43 UTC 2025

where maven:3.9.9-eclipse-temurin-8-alpine is the docker image and date the custom command that replaces the default mvn execution.

docker compose directive – command:

Being a decorator on top of docker’s base CLI docker compose can do anything that plain docker run can do. Thus we can do custom command executions here as well. In fact, it’s much more fun than using the default CLI. But thats just me. Now without further ado:

florian@ubuntu-jammy:~$ sudo docker compose up
[+] Running 1/1
 ✔ Container vagrant-maven-1  Created                                      0.0s 
Attaching to maven-1
maven-1  | Sat Mar  8 15:05:16 UTC 2025
maven-1 exited with code 0

And there is our today’s date.

What commands can we use in docker?

In this section I present my favorite commands to keep my docker containers open. The most intuitive is probably:

sleep infinity

That one is pretty self-explanatory. We just sleep for an infinite amount of time or until we manually cancel the process. The constant infinity is defined in the GNU world and serves us as a representation of positive infinity.

The next one is:

top

We just let your container dispay it’s realtime ressource usage thus keeping your container up and running. Time to docker exec into it and do our thing. Little caveat: Some images may strike top out to reduce the container’s size and to focus on it’s core business. If that’s the case, we should try my top-most favorite. This one should be available quite often and looks so cool:

tail -f /dev/null

That one waits on a neverending file and tries to spit out whatever it finds at the end of it – in this case dark and pure nothingness.

Final thoughts

This post was a personal matter of heart to me having had the use case several times. But since im getting old *sigh* I tended to forget what commands I can use. Now that’s finally a thing of the past. Give it a bookmark and it will keep reminding you as well. Do you have a favorite command to keep things up? Feel free to drop me a line in the comments down below. And if you still can’t get enough of Docker or Maven here are some recommendations just for you: My most recent star is this post about configuring TLS for HTTPS-secured Maven repos. And finally for the more docker-inclined people out there this post about docker ps – formatting might make your daily life a little simpler.

Have a sunny day!

Share it with: