Blogs on Personal weblog

Harvest Java Heap Dumps in constrained environments

Fri, 20 Jan 2023 20:40:30 +0100

Creating and analyzing Heap Dumps is a standard for analyzing memory issues with JVM based programs. To do this, you have to:

create the Heap Dump (duh!)
move the data to your computer
actually analyze the data

Each of these steps has it's own challenges though the first two seem trivial at first sight.

Creating the Heap Dump

The main challenge in Heap Dump creation I faced in today's production environments is that only essential software is installed in order to keep containers and the potential attack surface small. So likely there is only a JVM which is enough to run the program but not enough to get a Heap Dump. Through the recommendation of a colleague I became aware of jattach. It describes itself as:

Fixing the problem of too many tabs

Tue, 03 May 2022 21:22:29 +0200

Tab mayhem and windows inflation

How many browser tabs are open on your browser right now? Too many to count? Let me guess: Rows and rows of tabs belonging to different research tasks that you want to pick up later. The occasional tab on social network. That store you still have a cart open with that thing you maybe want to buy. Some are still relevant but many are not.

The last version manager you'll ever need

Mon, 18 Apr 2022 22:01:01 +0200

programming languages by AAMINE1965, licensed under CC BY-SA 4.0

Chances are high you have already used tfenv, pyenv, SDKMAN!, rbenv, nvm or some other version manager. In case you haven't: A version manager is a program that let's you install multiple versions of a tool for different contexts: Your one project needs version X, the other version Y.alpha. Just the tool in one version all the time does not cut it as different versions behave differently. So if people work on the same project with different versions of core tools, it often gets messy:

Why does MaxRAMPercentage stop working at heaps of ~ 30 GB?

Sat, 22 Jan 2022 18:30:50 +0100

The other day at work a Clojure program on a machine with 100 GB ran into OutOfMemory errors. A quick look at the metrics showed that the heap was only at about 30 GB though. Checking the JVM flags I found a configuration that is commonplace:

-XX:MaxRAMPercentage=80

So one would assume that the JVM could have sized the heap much larger than it actually did. Thanks to docker this behavior can be observed quite easily by doing a few tests like this one:

Create downtimes with Instance Refresh for EC2 Auto Scaling

Fri, 12 Nov 2021 10:12:13 +0200

When AWS launched Instance Refresh for EC2 Auto Scaling last year my colleagues and me were delighted: Should we be able to retire our half baked and mostly working lambda for restarting ec2 instances and hand this over to AWS Auto Scaling?

I think we were not the only ones looking forward to this: Auto Scaling groups is an extremely popular option of the most popular AWS service (EC2) of the biggest cloud provider in the world. At the same time all of these many customers are responsible to use a recent machine image with current security patches. That means refreshing instances in an Auto Scaling Group is something anybody out there needs to do on a regular basis¹ and anybody who does not like busywork will automate the process.

Do you ever forget to push your git commits?

Thu, 14 Oct 2021 15:23:33 +0200

The title of this post is a rhetorical question. At least I think it is. Or I like to think it is. What I can say with certainty is that I forgot my fair share of pushes. 😅

Sometimes I was the one tripping over my own mistake (Why is that commit not live? Ah, because the pipeline did not run. Because I did not push. facepalm). Sometimes my colleagues had to find out.☺️

Notifications for failed cron jobs

Thu, 14 Oct 2021 15:07:39 +0200

Every few years when I setup a new computer I stumble over the same question: "How do I get notified of failed cron jobs again?". It is pretty simple, but as I tripped over it again today, here is a little write-up so I do not have to google this again. Or potentially I will and then hopefully this post will turn up in the search results.😉

As cron is an old program from the days where the server use case was the predominant one, it will send emails with the output it's jobs will produce. Now I don't want my desktop machine to send emails over the internet to myself about broken jobs, when I am working on the very same machine at the same time.

Create virtualenvs in project directories

Sat, 17 Jul 2021 12:59:33 +0200

It's quite handy to create the virtual environment aka venv for a python project in the project folder itself. This means:

No huge .virtualenvs folder in your home directory with a lot of envs which you will never use again (The one on this computer is 1.2 GB big although I do not work with many Python projects at all(!)). If you are done with a project and delete all local files, the corresponding environment will be gone with it.
Pointing your editor/IDE to the current virtual env as well as deleting it is easy, because it is right there in the project root folder.

As there are many tools in this realm, obviously every one of them handles things a little bit differently and it is the default for none of the popular ones:

Why does poetry not use the specified Python version?

Fri, 16 Jul 2021 08:38:43 +0200

TL;DR: There is only one way to install poetry proper and that is with the installer script provided by the project itself.

I was sweating for the better part of an afternoon with a colleague about this. I had installed poetry, the new star in the python tooling world and it kept using the global version of Python instead the one for the project at hand, though poetry's own pyproject.toml specifies it. It was quite a frustrating experience.

Missing system modules in Python

Fri, 25 Jun 2021 22:22:59 +0200

I got to work on a Python code-base in the last days and have been struggling to get to a stable development environment. I learned some lessons along the way. Here is one of them, as it seems to be rather common and causes headaches for many people (a simple search has a whopping half a million hits on Google).

The error

ModuleNotFoundError: No module named '_lzma'

I found this error at the bottom of a very long stack trace after running a unit test. That was suprising as the tests had worked fine before. I stashed all current changes, but the error persisted. I recognized that the lzma module is part of the standard library. I could not find the module _lzma though. It turned out that the problem could be reproduced with a simple import statement:

Consume HTTP APIs lazily

Sat, 17 Apr 2021 13:26:00 +0200

TL;DR: Using lazy sequences for API consumption has advantages in memory consumption and promotes interactive development.

HTTP APIs are ubiquitous and consuming them is a programming task coming up quite regularly (at least for me). Therefore I found the following idiom pretty useful, which turns a paginated, remote data source (in this case the GitHub jobs API) into a lazy sequence:

(defn- fetch-lazy-jobs-seq!
  ([]
   (fetch-lazy-jobs-seq! 0))
  ([page]
   (let [jobs-url     (fn [page](format "https://jobs.github.com/positions.json?page=%d" page))
         {body :body} (http/get (jobs-url page) {:as :json})]
     (if (seq body)
       (lazy-cat body (fetch-lazy-jobs-seq! (inc page)))
       body))))

In the REPL it is comfortable to just inspect parts of the data without waiting for many HTTP requests to be send:

Making REPL driven development easy with leiningen

Tue, 13 Apr 2021 20:40:19 +0200

TL;DR: Leiningen provides facilities that make using a REPL driven workflow a breeze. You should consider using them on your projects for a more pleasant development experience.

Make your system reloadable

First of all the application should be reloadable, that means restartable in the REPL without restarting the entire process. If the app can shutdown and start quickly, this makes development way faster, less error prone and more pleasing experience. If you use component, this can be easily done with the reloaded.repl library provided by weavejester.

Clone all those gitlab repositories

Sat, 13 Jul 2019 20:40:19 +0200

If you change work environments once in a while you know that there is one thing that you are doing quite a lot of when starting off: Cloning quite a lot of repositories. Microservices, git and infrastructure as code among other things have let to a big number of repositories in almost all organizations.

As I was doing that the last time it got really tedious. Mainly because I wanted to search the whole code base and that did not work with the local gitlab instance. Naturally I turned to the shell to give this a shot and as it turns out it is a one liner (line breaks added for your convenience ;)), mostly thanks to python-gitlab and the inherent awesomeness of the shell:

Thinkpad Ultra Dock Annoyances

Thu, 08 Mar 2018 22:26:18 +0200

Just a few days ago I finally fixed an annoyance with my Thinkpad T460p and the Ultra Dock: Removing the laptop from the Dock results in it going into hibernation. Then after opening the lid it does not switch the built in screen back on. This turns out to be easy to fix.

Undocking and docking leads to a lot of events which can be observed by a simple

sudo acpi_listen

The event that seems to be the most promising is used as a trigger to correct the display settings. In my case that meant putting

Releasing clojure libs

Sat, 15 Jul 2017 22:44:39 +0200

Releasing libraries is a tedious but important task. There is lots to do and lots to screw up:

Tagging the release state
Making a clean build
Signing the artifacts
Pushing them to some repository
Bumping the version to the next snapshot iteration
And so on and so forth…

Having a dark past in the java world I looked for something like the maven release plugin for leiningen, which automates the whole process quite nicely. Turns out there exist a lot of release plugins. Right now clojars lists 50 results for the search term release.

Create bugs with overtone

Fri, 14 Jul 2017 22:26:18 +0200

These days I stumbled upon some code committed by a very competent but sadly no longer present colleague and was baffled to see this:

catch (Throwable t (log/error t "horrible things happend))

This left me puzzled and after looking for clues why this had been done (and not finding any) I decided to remove this exception dodger and move on. Turns out I created a bug.

Why? Because this code was called in a job scheduler from the very popular overtone/at-at library. And it showed the following behavior: