Create bugs with overtone


These days I stumbled upon some code committed by a very competent but sadly no longer present colleague and was baffled to see this:

catch (Throwable t (log/error t "horrible things happend))

This left me puzzled and after looking for clues why this had been done (and not finding any) I decided to remove this exception dodger and move on. Turns out I created a bug.

Why? Because this code was called in a job scheduler from the very popular overtone/at-at library. And it showed the following behavior:

(require '[overtone.at-at :as at])
(def counter (atom 0))
(at/every 100 #((swap! counter inc) (throw (Exception. "boom"))) (at/mk-pool))
(Thread/sleep 1000)
@counter
=>1

I found that behavior rather surprising: If a job fails one time that does not mean it should never be executed again. Plus if you do not have this in place you are never going to see any trace of this unsuspected exception in the logs and your scheduled job is just not running anymore without any indication as to why this is happening.

So what to do? The best thing I could think of was a wrapper:

(defn exception-to-log [f & args]
  (try (apply f args)
       (catch Exception e
         (log/error e "Exception caught in order to keep it schedule."))))

So when using overtone/at-at I recommend to put something like this in place.


See also