Why does MaxRAMPercentage stop working at heaps of ~ 30 GB?

The other day at work a Clojure program on a machine with 100 GB ran into OutOfMemory errors. A quick look at the metrics showed that the heap was only at about 30 GB though. Checking the JVM flags I found a configuration that is commonplace: -XX:MaxRAMPercentage=80 So one would assume that the JVM could have sized the heap much larger than it actually did. Thanks to docker this behavior can be observed quite easily by doing a few tests like this one: [Read More]
jvm  heap  memory 

Create downtimes with Instance Refresh for EC2 Auto Scaling

When AWS launched Instance Refresh for EC2 Auto Scaling last year my colleagues and me were delighted: Should we be able to retire our half baked and mostly working lambda for restarting ec2 instances and hand this over to AWS Auto Scaling? I think we were not the only ones looking forward to this: Auto Scaling groups is an extremely popular option of the most popular AWS service (EC2) of the biggest cloud provider in the world. [Read More]
aws  ec2 

Do you ever forget to push your git commits?

The title of this post is a rhetorical question. At least I think it is. Or I like to think it is. What I can say with certainty is that I forgot my fair share of pushes. 😅 Sometimes I was the one tripping over my own mistake (Why is that commit not live? Ah, because the pipeline did not run. Because I did not push. facepalm). Sometimes my colleagues had to find out. [Read More]
git  linux 

Notifications for failed cron jobs

Every few years when I setup a new computer I stumble over the same question: "How do I get notified of failed cron jobs again?". It is pretty simple, but as I tripped over it again today, here is a little write-up so I do not have to google this again. Or potentially I will and then hopefully this post will turn up in the search results.😉 As cron is an old program from the days where the server use case was the predominant one, it will send emails with the output it's jobs will produce. [Read More]
cron  linux 

Create virtualenvs in project directories

It's quite handy to create the virtual environment aka venv for a python project in the project folder itself. This means: No huge .virtualenvs folder in your home directory with a lot of envs which you will never use again (The one on this computer is 1.2 GB big although I do not work with many Python projects at all(!)). If you are done with a project and delete all local files, the corresponding environment will be gone with it. [Read More]

Why does poetry not use the specified Python version?

TL;DR: There is only one way to install poetry proper and that is with the installer script provided by the project itself. I was sweating for the better part of an afternoon with a colleague about this. I had installed poetry, the new star in the python tooling world and it kept using the global version of Python instead the one for the project at hand, though poetry's own pyproject. [Read More]

Missing system modules in Python

I got to work on a Python code-base in the last days and have been struggling to get to a stable development environment. I learned some lessons along the way. Here is one of them, as it seems to be rather common and causes headaches for many people (a simple search has a whopping half a million hits on Google). The error ModuleNotFoundError: No module named '_lzma' [Read More]

Consume HTTP APIs lazily

TL;DR: Using lazy sequences for API consumption has advantages in memory consumption and promotes interactive development. HTTP APIs are ubiquitous and consuming them is a programming task coming up quite regularly (at least for me). Therefore I found the following idiom pretty useful, which turns a paginated, remote data source (in this case the GitHub jobs API) into a lazy sequence: (defn- fetch-lazy-jobs-seq! ([] (fetch-lazy-jobs-seq! 0)) ([page] (let [jobs-url (fn [page](format "https://jobs. [Read More]

Making REPL driven development easy with leiningen

TL;DR: Leiningen provides facilities that make using a REPL driven workflow a breeze. You should consider using them on your projects for a more pleasant development experience. Make your system reloadable First of all the application should be reloadable, that means restartable in the REPL without restarting the entire process. If the app can shutdown and start quickly, this makes development way faster, less error prone and more pleasing experience. [Read More]

Clone all those gitlab repositories

If you change work environments once in a while you know that there is one thing that you are doing quite a lot of when starting off: Cloning quite a lot of repositories. Microservices, git and infrastructure as code among other things have let to a big number of repositories in almost all organizations. As I was doing that the last time it got really tedious. Mainly because I wanted to search the whole code base and that did not work with the local gitlab instance. [Read More]
gitlab  git  bash