Q:

Timing analysis of Clojure code

Hello Clojure experts, 
I am trying to do some timing tests in Clojure 1.3 and I thought I'd ask a question based on an existing piece of code that solves a differential equation that is adapted from  this blog post.
Code follows: 
;; the differential equation is                                                                                            
;; dy/dt = f(y,t) = t - y                                                                                                  

(defn f [t y] (- t y))

;; solve using Euler's method                                                                                              
(defn solveEuler [t0 y0 h iter]
  (if (> iter 0)
    (let [t1 (+ t0 h)
          y1 (+ y0 (* h (f t0 y0)))]
      (recur t1 y1 h (dec iter)))
    [t0 y0 h iter]))

(defn multipleSolveEuler []
  (let [steps '(1 10 100 1000 10000 100000)
        results (map #(second (solveEuler 0.0 0.0 (/ 1.0 %) %)) steps)
        errors  (map #(- (Math/exp -1) %) results)]
    (partition 3 (interleave steps results errors))))

(def *cpuspeed* 2.0)

(defmacro cyclesperit [expr its]
  `(let [start# (. System (nanoTime))
         ret# ( ~@expr (/ 1.0 ~its) ~its )
         finish# (. System (nanoTime))]
     (int (/ (* *cpuspeed* (- finish# start#)) ~its))))

(defn solveEuler-2 [t0 y0 h its]
  (let [zero (int 0)]
    (loop [t0 (double t0), y0 (double y0), h (double h), its (int its)]
      (if (> its zero)
        (let [t1 (+ t0 h)
              y1 (+ y0 (* h (- t0 y0)))]
          (recur t1 y1 h (dec its)))
        [t0 y0 h its]))))

So when I say 
(time solveItEuler-2 0.0 1.0 (/ 1.0 1000000000) 1000000000))

I get a time 6004.184 msecs on a 6 month old Macbook pro. I issue the command again, and I get around the same time. But when I run it 3 other times I get times in the range of 3500 msec. I have noticed this before for other code snippets, and was wondering why this is so. I suppose I would expect a roughly similar execution time across successive runs.
Is it my lack of understanding how "time" works, something I am missing, or some kind of caching happening under the hood?
Thanks.

A:

It's not time that's relevant, but rather the JVM optimizing the code at runtime.  What you observe is typical; execution time will drop and then stabilize after about 3 invocations.

A:

I recommend using the Criterium library for benchmarking. It is designed to deal with the pitfalls of benchmarking code that runs on the JVM.

