Trying to grok transducers one step at a time


Hi there, I am Clojure newbie trying to understand some concepts behind transducers. I went over the first video on Transducers series from Timothy Baldrige. And there is a simple comp function I don’t understand at the end of the video. But I’ll get to this issue shortly. First I want to write what I have learned about transducers so far, so this thread can be of use to other newbies like me trying to learn Clojure.

So let’s start:

; we have some vector of numbers from 0 - 9 which represents our data
(def data (vec (range 10)))

; Now lets think about map and filter functions.  
; We can implement those ourselves using  a reduce function.

(def -map [f coll]
    (fn [acc v]
      (conj acc (f v)))))

; Now doing something like this should work fine

(-map inc [1 2 3])

; And we can also implement filter which 
; is very similar to our -map function
; we just have to add if condition

(defn -filter [f coll]
    (fn [acc v]
      (if (f v)
        (conj acc v)
    [] coll))

; And now we can check that our -filter 
; works as expected

(-filter odd? data)
;=> [1 3 5 7 9]

; You can even chain those functions 
; together and it should work just fine.

(->> data
  (-map inc)
  (-filter odd?))

; But if you look at our -map and -filter solutions, 
; you can see, that both have reducing logic 
; and it would be nice if we could somehow 
; pull it out from those functions.
; So let us create a function mapping

(defn -mapping [f]
  (fn [acc v]
    (conj acc (f v))))

; the above function receives a function that is used 
; on single element when reducing over some collection
; or logical sequence and it returns a reducing function 
; that is used by reduce function.
; Now we can call reduce like this

(reduce (-mapping inc) [] data)

; Great, this works. But we still have one problem. 
; Its the conj function. This is the place 
; where we actually say how
; the resulting sequence should be put together. 
; But we dont want that, 
; because we are assuming too much here, we 
; are assuming that the sequence we are reducing over is conjable. 
; (it knows how to conj)
; We need to decomplect this part, 
; so  that we create a function that receives
; the job (the function/the how part) as a parameter.
; We name it xf, it stands for "transforming function" 
; and in turn returns our reducing 
; function that uses our xf internally

defn -mapping2 [f]
  (fn [xf]
    (fn [acc v]
      (xf acc (f v)))))

; create our function that returns a function 
; which expects the xf part as a parameter

(def rfn (-mapping2 inc))

; then use our function to create a reducing function. 
; As you can see, we pass the conj function as a parameter
; to rfn, which returns our reducing function

(reduce (rfn conj) [] data)

; we can do the same with filter, 
; we abstract away the xf operation 
; like we did in -mapping2

(defn -filtering2 [f]
  (fn [xf]
    (fn [acc v]
      (if (f v)
        (xf acc v)

(def rfn2 (-filtering2 odd?))

(reduce (rfn2 conj) [] data)

; we can now even compose the reducing functions, 
; and this is the part that gets confusing to me

(def rfn3 (comp
             (-mapping2 inc)
             (-filtering2 odd?)))

; As I know. The comp function firstly evaluates 
; the last expression inside, so the (-filtering2 odd2?) is evaluated,
;  which returns a (fn [xf]...) which receives the conj function 
; and returns a (fn [acc v]...) which comes as an input 
; to what (-mapping2 inc) returns ?? 
; But (-mapping2 inc) returns a fn that expects xf as a parameter also.

(reduce (rfn3 conj) [] data)
;=> [1 3 5 7 9]


Oh, I think I got it now why comp is working. :slight_smile: So first conj is passed to what (-filterring odd?) returns, which in turn returns the filtering reducing function (fn [acc v]…), which is passed as a xf into what (-mapping2 inc) returns and then it’s executed on the last line of -mapping2 as (xf acc (f v)) which is actually ((fn [acc v]…) acc (f v))
In short, this works, because the two reducing functions both take two arguments.

Sometimes staring works :blush:


Hey @smnplk!

I’m glad you answered your own question! I’m sorry I couldn’t answer it more quickly.

Rock on!


Hey Eric !

No problem, sometimes it’s better to figure things out on your own :slight_smile: And btw, thanks for putting this great community together. I would love more people to join though.