Efficiently debugging functions


#1

Coming from Node.js & front-end development, I’m used to working on a function by putting a debugger in the middle of it. I can open the Chrome inspector tools and see what intermediate variables are set at and test new code in the inspector with the current context. For example:

function doSomething(a, b) {
  const aa = transformA(a)
  const bb = transformB(b)
  debugger  // stop to make sure aa and bb were properly returned (or remember what is returned)
  const cc = _.reduce(bb, (memo, val, key) => {
    // doing some code
    debugger    // inspect the built-up value (memo) interactively 
    return memo
  }, {})
  return cc
}f

There are lots of very experienced Clojure developers who never use a debugger and swear by the REPL. I’m ready to do that, but how do folks do this efficiently? Stealing examples from Tonsky’s blog (http://tonsky.me/blog/readable-clojure/), how would I access intermediate values as I build up these functions:

(defn created-at [db a v]
  (let [datoms (d/datoms db :avet a v)
        datom  (first datoms)
        tx-id  (:tx datom)
        tx     (d/entity db)]
    ;;I want to inspect my let vars here to make sure I did them correctly
    (:db/txInstant tx)))


(->> users
  (filter :age)
  (remove :fired)
  (my-custom-function)
  ;; I want to make sure my return value to the next function is correct 
  ;; How do I stop the world *here* and inspect what's going on?
  (map #(update % :age inc)))

If stopping and inspecting in the middle of a chain isn’t done, how do people effectively write non-trivial functions? For the examples above, you would have to be confident in each of your let or threading macro steps. Or create a separate function that just outputs things you are interested in, but that feels clunky.

I would love to get some ideas on this. Thanks


#2

Hi @jefffriesen,

Great question!

So, there are two answers, depending on when you happen to find yourself.

If you’re building a new function and adding steps to the let, you can do it incrementally. Start with the first one:

(defn created-at [db a v]
  (let [datoms (d/datoms db :avet a v)]
    (prn datoms)))

Then add the next and change the println:

(defn created-at [db a v]
  (let [datoms (d/datoms db :avet a v)
        datom  (first datoms)]
    (prn datom)))

If you’re adding to a threading chain, it’s pretty obvious.

The other time is if you have an existing function and you want to inspect some value in the let.

(defn created-at [db a v]
  (let [datoms (d/datoms db :avet a v)
        datom  (first datoms)
        tx-id  (:tx datom)
        tx     (d/entity db)]
    ;; just print out everything you want to know here:
    (prn datoms)
    (prn datom)
    (prn tx-id)
    (prn tx)
    (:db/txInstant tx)))

Then delete those prints when you’re satisfied. But! What happens if stuff breaks along the way. Let’s say the third let binding throws an exception, so you never get to the print statements. You could break the let in two, but that’s a pain. So here’s a little trick I use (and I’m not alone!).

(defn created-at [db a v]
  (let [datoms (d/datoms db :avet a v)
        _ (prn datom)
        datom  (first datoms)
        _ (prn datom)
        tx-id  (:tx datom)
        tx     (d/entity db)]
    (:db/txInstant tx)))

You can put prints in the binding but just use a dummy _ variable. Not recommended for production code but totally fine during development.

As for printing stuff in the middle of a chain, there’s another trick. You can’t use prn because it returns nil and breaks the rest of the chain. But you can use doto. doto let’s you call functions on a value in order, but it always returns the value itself. So if I wanted to see what it looks like after filtering and after the my-custom-function call:

(->> users
  (filter :age)
  (doto prn)
  (remove :fired)
  (my-custom-function)
  (doto prn)
  (map #(update % :age inc)))

That’s what I do. I probably should use a debugger for stuff like this. I find debuggers kind of a pain because they’re all different and I’ve never learned one well enough to get really good at it. I always find that I have to click step so many times to get where I want to go. But that’s probably just me not taking the time to understand their features. CIDER has a debugger which I have never used. I’ve gotten used to the debugger in Firebug, but I don’t find that I miss it.

There’s another thing that happens, when you have a live system that hot reloads, is you can inspect stuff much more readily and quickly by simply exposing it as part of the UI/printed lines. What I tended to do in Java or JavaScript, because it wasn’t live, was to write a lot of code, run it and see that it didn’t work, then put some breakpoints in to figure out where it went wrong. In Clojure/ClojureScript, I write much less code before I see that it’s doing what I expect. So I’m rarely in a place to practice using a debugger :slight_smile:

I’m also curious, what editor are you using for Clojure?


#3

These are great tips Eric. Thank you. I’m definitely going to try that.

Based on your recommendation, I was using Nightlight for the tutorials. The InstaREPL is really nice. But when I tried to use it for my own code it was pretty clunky. If there were errors in the code when I started it up I couldn’t run the editor. I couldn’t add directories. Sometimes errors wouldn’t go away even when I thought they were fixed but would disappear when I restarted it. So then it’s hard to know if something I’m doing is really fixing it or I need to restart the editor. And it didn’t have useful keyboard shortcuts.

I’m now using Atom + Proto REPL. I do use the Auto Eval function (Insta REPL). In theory the whole package together is great but it still has problems. I’m constantly mixing up what namespace is in the main REPL and when I need to reload it. I’m constantly getting errors about namespaces and something is not defined in the REPL namespace. It’s hard - some errors are obvious and many aren’t. Is this a REPL problem that’s easily fixed by hitting a keybinding or a real problem with my code. But that’s not a new problem with Atom. I had that same problem in Cursive when I was trying it.

There has been a lot of questions by experienced Clojurists over the last few months about what is keeping beginners from using Clojure. One of the big obstacles has been the question I just posted (and you answered well) and related, the REPL. Everyone claims how it’s the best part of REPL and it is easy when you show them that you can do (+ 1 1) and get an answer. But when actually trying to build real code I hit a wall. It’s possible that editors could do better - auto-refreshing the REPLs namespace, GUI controls to change namespaces, better error messages. But in the meantime, maybe education.

The biggest thing I would want from a Purelyfunctional.tv tutorial is how in the real world do you use the REPL and debugging (whether that is a debugger or prn statements). It was so helpful when you explained in the Intro to Clojure series how to read the JVM stack traces.

Thanks for your help.


#4

Hi Jeff,

Eric had some good points. One trick I like to make debugging is to use the spy and spyx commands from the Tupelo Library. This allows you to write code like so:

(-> 30
    (+ 4)
    (spy :dbg)
    (* 10))
; :dbg => 34            ; integer result = 34
340

or this:

(it-> 1                 ; tupelo.core/it->
      (spyx (inc it))
      (* 2 it))
; (inc it) => 2     ; the expression is used as the label
4

Another variant is useful when you want to print out the type of the value as well:

(defn mystery-fn [] (into (sorted-map) {:b 2 :a 1}))
(spyxx (mystery-fn))
;  (mystery-fn) =>  <#clojure.lang.PersistentTreeMap {:a 1, :b 2}>"

I would also strongly recommend using the lein plugin test-refresh. I think this is much better than a REPL since you can do real editing in your favorite editor/IDE (not just command-line typing), and save your work as unit or regression tests. test-refresh will auto-reload your code upon each file save, then re-run any changed test files. I find myself coding in the test namespace, getting matching code & tests to work, then migrating the code into the core ns once it is developed & tested & organized the way I like it. A big time saver!


#5

@cloojure - that’s a great tip. I also can use a couple other string and csv functions from your library. Thank you.