Kategorie: Q3A

Quick Question – Quick Answer

Veröffentlicht am

AccessDenied in psutil

psutil is an interesting Python package that provides us with valuable insights about running processes, their memory and CPU usage and many more key aspects for monitoring and profiling processes across all major platforms. Thus, it is an incredibly useful tool for system admins, developers and testers alike. Now one of my favourite methods is cmdline(). Defined in the package’s Process class, it yields the whole CLI command of a running process in array form. There is one important catch on Windows though that got me kinda by surprise: While iterating casually through all my processes I suddenly got an AccessDenied error.

What did I try?

For an advanced verification in a process management – related test case I wanted to print the executed CLI command in a log file, so basically this:

import psutil

for pc in psutil.process_iter():
    print(pc.cmdline())

But that will just throw a painful AccessDenied error at us. Uff.

Why did that happen?

The reason is that psutil is quite consequent: It really lists every running process. That means even SYSTEM or root processes . That’s okay, it might even be interesting in one case or another, and you can still access selected attributes like the name() of a root process, if you want. But for me, it doesn’t suffice. I want to see the full-fledged cmdline(), but I understand that SYSTEM processes are none of my business. Once I came to term with that fact, the solution was easy: We just skip them.

The solution

What I did was applying a try-except around the loop’s inner statement:

import psutil

for pc in psutil.process_iter():
    try:
        print(pc.cmdline())
    except psutil.AccessDenied:
        continue

The continue statement will make sure that the processes that I’m not supposed to see are happily skipped without hurting the rest of the program flow.

But what if I need to monitor foreign processes?

In that case, we would need to execute the script within the process owner’s user context. That might be a bit fiddly in Windows depending on the use case, but of course that’s still possible. Just remember to keep the try-except block, because there still will be processes you wouldn’t be allowed to see.

Mine, for example. 😄

Conclusion

So far for today. I hope this little Q3A (quick question quick answer) could shed some light upon that surprising AccessDenied error. If you want to learn more about psutil, I strongly recommend the readthedocs page. Otherwise, if you want to see more quick tips, I have one more for Python here. That one covers Python’s environment variable handling. As an alternative, if you are – like me – into containers, here is a handy docker ps trick useful for monitoring tasks as well.

Happy coding!

Share it with:
Veröffentlicht am

Python Environment Variables: getenv() vs. environ[ ]

Last week, I was about to execute a test run using one of our functional test suites that requires a certain environment variable – and I forgot to set it, whoops.. The answer was harsh and generic: A good ol‘ Python KeyError. That put me up with two questions:

  1. What options do I have to get environment variables with Python and
  2. which one is the best?

Let’s start checking them out.

Introducing os.environ[key]

The first option is a simple dictionary that is prefilled with all environment variables when starting the Python process. This provides us with a simple-to-use interface that we already know and love.

Let’s try it out:

For python environment variables with os.environ[key], do the following: export MY_VAR="test", python3 -i, import os and os.environ["MY_VAR"]. This should yield 'test'.
Python environment variables with os.environ[key]

But what happens, when we query a variable that is not set? Then we hopefully prepared our try-except Block, because that call is going down the river faster than sound. This happens because dictionaries throw generic KeyErrors, when the queried key could not be found in the dict. That’s what happened in my test run.

os.getenv(key, default=None)

The second option follows a more higher-level approach by providing a function that takes our key in question and an optional default value. The usage is straight forward and it won’t crash as harshly as the first option does.

Let’s try it out:

For python environment variables with os.getenv(key, default), do the following: export MY_VAR="test", python3 -i, import os and os.getenv("MY_VAR"). This should yield 'test'. If you query MY_VAR2 instead, it should yield None. If you do os.getenv("MY_VAR2", "default"), it should yield default.
Python environment variables with os.getenv and default value handling

As we can see, we can now use any key we want; worst case is a result of None. But the increased flexibility comes with a price: We have to take care of whatever the function returns. Even harder, due to the fact that the default defaults to None (pun semi-intended), this solution is prone to hidden bugs. Therefore use it with care and set an appropriate default value if possible.

Conclusion: Which one works best for you?

We saw both options now, but which one works best (for you)? At the end of the day, it comes down to preference regarding two factors:

  1. Do you like your code to be more low-level or high-level?
  2. Do you want your fails fast or more controlled?

This is a decision you have to make, but once you have it, you find everything you need to get your environment variables with Python right here.

So long for the quick journey into my Python life. Usually I talk more about Rust as seen here and here, but since Python is my language on the job, there will definitely be more coming up soon. So if you are a friend of british comedy, feel free to stay tuned. Additionally, since container technologies are getting more and more important in my day job as well, I will write more about these techs, too, starting with this little trick.

Happy holidays and a merry Christmas Eeve! 🎄

Share it with: