Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Wednesday, December 17, 2014

Working Effectively with Unit Tests Official Launch

Today marks the official release release of Working Effectively with Unit Tests. The book is available in various formats:
I’m very happy with the final version. Michael Feathers wrote a great foreword. I incorporated feedback from dozens of people - some that have been friends for years, and some that I’d never previously met. I can’t say enough great things about http://leanpub.comand I highly recommend it for getting an idea out there and making it easy to get fast feedback. 

As far as the softcover edition, I had offers from a few major publishers, but in the end none of them would allow me to continue to sell on leanpub at the same time. I strongly considered caving to the demands of the major publishers, but ultimately the ability to create a high quality softcover and make it available on Amazon was too tempting to pass up.

The feedback has been almost universally positive - the reviews are quite solid on goodreads (http://review.wewut.com). I believe the book provides specific, concise direction for effective Unit Testing, and I hope it helps increase the quality of the unit tests found in the wild.

If you'd like to try before you buy, there's a sample available in pdf format or on the web.


Wednesday, May 21, 2014

Working Effectively with Unit Tests

Unit Testing has moved from fringe to mainstream, which is a great thing. Unfortunately, as a side effect developers are creating mountains of unmaintainable tests. I've been fighting the maintenance battle pretty aggressively for years, and I've decided to write a book that captures what I believe is the most effective way to test.

From the Preface

Over a dozen years ago I read Refactoring for the first time; it immediately became my bible. While Refactoring isn’t about testing, it explicitly states: If you want to refactor, the essential precondition is having solid tests. At that time, if Refactoring deemed it necessary, I unquestionably complied. That was the beginning of my quest to create productive unit tests.

Throughout the 12+ years that followed reading Refactoring I made many mistakes, learned countless lessons, and developed a set of guidelines that I believe make unit testing a productive use of programmer time. This book provides a single place to examine those mistakes, pass on the lessons learned, and provide direction for those that want to test in a way that I’ve found to be the most productive.
The book does touch on some theory and definition, but the main purpose is to show you how to take tests that are causing you pain and turn them into tests that you're happy to work with.

For example, the book demonstrates how to go from...

looping test with many (built elsewhere) collaborators
.. to individual tests that expect literals, limit scope, explicitly define collaborators, and focus on readability
.. to fine-grained tests that focus on testing a single responsibility, are resistant to cascading failures, and provide no friction for those practicing ruthless Refactoring.
As of right now, you can read the first 2 chapters for free at https://leanpub.com/wewut/read

I'm currently ~25% done with the book, and it's available now for $14.99. My plan is to raise the price to $19.99 when I'm 50% done, and $24.99 when I'm 75% done. Leanpub offers my book with 100% Happiness Guarantee: Within 45 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks. Therefore, if you find the above or the free sample interesting, you might want to buy it now and save a few bucks.

Buy Now here: https://leanpub.com/wewut

Monday, May 19, 2014

Weighing in on Long Live Testing

DHH recently wrote a provocative piece that gave some views into how he does and doesn't test these days. While I don't think I agree with him completely, I applaud his willingness to speak out against TDD dogma. I've written publicly about not buying the pair-programming dogma, but I hadn't previously been brave enough to admit that I no longer TDD the vast majority of the time.

The truth is, I haven't been dogmatic about TDD in quite some time. Over 6 years ago I was on a ThoughtWorks project where I couldn't think of a single good reason to TDD the code I was working on. To be honest, there weren't really any reasons that motivated me to write tests at all. We were working on a fairly simple, internal application. They wanted software as fast as they could possibly get it, and didn't care if it crashed fairly often. We kept everything simple, manually tested new features through the UI, and kept our customer's very happy.

There were plenty of reasons that we could have written tests. Reasons that I expect people will want to yell at me right now. To me, that's actually the interesting, and missing part, of the latest debate on TDD. I don't see people asking: Why are we writing this test? Is TDD good or bad? That depends; TDD is just a tool, and often the individual is the determining factor when it comes to how effective a tool is. If we start asking "Why?", it's possible to see how TDD could be good for some people, and bad for DHH.

I've been quietly writing a book on Working Effectively with Unit Tests, and I'll have to admit that it was really, really hard not to jump into the conversation with some of the content I've recently written. Specifically, I think this paragraph from the Preface could go a long way to helping people understand an opposing argument.

Why Test?

The answer was easy for me: Refactoring told me to. Unfortunately, doing something strictly because someone or something told you to is possibly the worst approach you could take. The more time I invested in testing, the more I found myself returning to the question: Why am I writing this test?

There are many motivators for creating a test or several tests:
  • validating the system
    • immediate feedback that things work as expected
    • prevent future regressions
  • increase code-coverage
  • enable refactoring of legacy codebase
  • document the behavior of the system
  • your manager told you to
  • Test Driven Development
    • improved design
    • breaking a problem up into smaller pieces
    • defining the "simplest thing that could possibly work"
  • customer approval
  • ping pong pair-programming
Some of the above motivators are healthy in the right context, others are indicators of larger problems. Before writing any test, I would recommend deciding which of the above are motivating you to write a test. If you first understand why you're writing a test, you'll have a much better chance of writing a test that is maintainable and will make you more productive in the long run.

Once you start looking at tests while considering the motivator, you may find you have tests that aren't actually making you more productive. For example, you may have a test that increases code-coverage, but provides no other value. If your team requires 100% code-coverage, then the test provides value. However, if you team has abandoned the (in my opinion harmful) goal of 100% code-coverage, then you're in a position to perform my favorite refactoring: delete.
I don't actually know what motivates DHH to test, but if we assumed he cares about validating the system, preventing future regressions, and enabling refactoring (exclusively) then there truly is no reason to TDD. That doesn't mean you shouldn't; it just means, given what he values and how he works, TDD isn't valuable to him. Of course, conversely, if you value immediate feedback, problems in small pieces, and tests as clients that shape design, TDD is probably invaluable to you.

I find myself doing both. Different development activities often require different tools; i.e. Depending on what I'm doing, different motivators apply, and what tests I write change (hopefully) appropriately.

To be honest, if you look at your tests in the context of the motivators above, that's probably all you need to help you determine whether or not your tests are making you more or less effective. However, if you want more info on what I'm describing, you can pick up the earliest version of my upcoming book. (cheaply, with a full refund guarantee)

Tuesday, April 02, 2013

Emacs Lisp: Find Java Sources

Confession: I really hope someone can tell me I'm doing this wrong. I can't believe there isn't an easier way.

I work with Clojure, in Emacs, almost every day. Navigating the source is usually fairly easy. If I want to navigate to a function definition, all I need to press is M-., and if I want to navigate back, M-, does the trick. This works for Clojure that I've written, as well as Clojure that lives in the libraries that I reference. That's fine the vast majority of the time, but occasionally I need to navigate to the Java source of some library I'm using. This is where I can't believe that no one else has solved this problem.* If I'm in a clj file, my cursor is on a Java class, I don't know of any way to easily navigate to the class definition.

Context: I use fig for dependency management (at work). I have my projects set to put sources in ./the-project/lib/sources; therefore, the following solution assumes the sources are in that directory. If you use Lein (for deps), I'm sure the sources are on your local drive somewhere. All you need to do is change the lisp below to point to your source dir.

Additionally, I use Lein, so I have a project.clj at the root of my project; however, there's nothing special or required about "project.clj". You could just as easily put a this.is.a.project.root file in the root of your project, and search for that file.

The following code will search the your source jars for a Java class, and open the first match that it finds (or no match, if no match is found)

disclaimer, I'm still an emacs lisp beginner.

The previous code is pretty straightforward. Line 3 uses expand region to mark whatever Java class my cursor is currently on or near. You could type the word instead if you like, this page should help you understand how.
Line 4 and 6 find and verify where the project root lives.
Line 8 (and 9) greps for a string (the Java class) in a directory (my source dir).
Line 10 switches to the grep results.
Line 11 sleeps, waiting for the grep results to return.
Line 12 searches forward, looking for the first match.
Line 13 opens the jar of the first match.
Line 14 assigns the current point to a var named 'current-point'.
Line 15 searches forward to the end of the jar name.
Line 16 grabs the name of the jar and switches to that buffer.
Line 17 searches the jar's dired buffer for the name of the class.
Line 18 opens the first class name match.
(Line 19 lets you know if your project root could not be determined)

That's it, you can now easily find java source (with the occasional conflicting name annoyance). It's not pretty, but it gets the job done.

* I've been told that a few Java modes are good, if I can easily use those to navigate from my Clojure to Java, please leave a link to a manual I can dig into. I assume there are etags solutions, but it's not clear what the best way to go is. I'm sure there's an easy solution for navigating Java from Clojure, I'm just having a hard time finding it.

Tuesday, October 09, 2012

Java: Add A Println To A 3rd-Party Class Using IntelliJ's Debugger

When I'm working with 3rd party Java code (e.g. joda.time.DateTime) and I want to inspect values that are created within that code, I generally set a breakpoint in IntelliJ and take a look around. This works the vast majority of the time; however, there are times when this approach isn't an option. For example, if stopping your application changes the state of what you're looking at, or if stopping the application causes your breakpoint to fire repeatedly in irrelevant situations, then you'll probably want another solution for looking around 3rd party code.

It turns out, IntelliJ gives you the ability to inspect values within 3rd party code - without needing to stop the application at a breakpoint. The following code uses the 3rd party library joda-time, and is simply printing the current time to the console.

note: this code could simply use a breakpoint if you wanted to inspect some 3rd party code, but I didn't see the value in creating a more complicated example.

In the example, we're calling the constructor of DateTime, which simply delegates to the BaseDateTime constructor, which can be seen in the following screenshot.

So, we're looking at joda-time code that we can't edit, but our task is to get the value of DateTimeUtils.currentTimeMillis() and ISOChronology.getInstance() without stopping the application. The solution is to add a breakpoint, but do a bit of customization.

In the following screenshot we've added a breakpoint to the line that contains the values we're interested in and we're right clicking on the breakpoint, which will allow us to edit the properties of the breakpoint.

When we open the properties of a breakpoint, they will look something like the following screenshot.

In the properties window you'll want to set Suspend policy to 'None', and you'll want to set something in Log evaluated expression. In my properties window I've set the following expression: System.out.println(DateTimeUtils.currentTimeMillis() + ", " + ISOChronology.getInstance()); - though, you can't see the full snippet in the screenshot.

At that point you close the properties pop-up and run your app once again (in Debug mode).

As you can see from the output, both the millis and the ISOChronology were printed to the console, and the application processing was never broken (it simply ended as expected).

That's it - now you can println in code you don't own to your heart's content.

Wednesday, June 13, 2012

So, You Dropped Out of College

I was recently chatting to some friends about all the apprenticeship experiments going on and (name withheld to protect the innocent) lamented:
So, what happens when they quit that first job (or worse, get laid-off) and their only skill is writing a Rails app?
I know what happens. It happened to me.

Okay, I didn't actually participate in an apprenticeship program that taught me Rails. However, I did drop out of college after gaining some experience programming, get a job building web applications, and I was laid off about a year later. Getting another job wasn't so hard, but it was hard to find a job that paid as well. It turns out, there's not a huge demand for people who don't have formal education or much experience, go figure.

While I was looking for a new job I noticed that there were significantly more jobs for people with certain skills. I'm sure it differs by location and time - in Jacksonville FL, around 2000, 75% of the jobs openings were looking for people with Microsoft .net skills. Sadly, I did not have .net skills. That was when I learned that I needed to have not just the skills necessary to do my job today, but also the skills necessary to land a new job tomorrow. Since I wasn't already in that situation, I had to settle for a job making .5 of my previous salary; however, my new job did allow me to learn .net.

I firmly believe: if you have less than 4 years of experience, you should always know what technology is in the highest demand for your area, and you should be proficient enough that you can do well in an interview. Perhaps your day to day job already teaches you the most highly sought after skills, fantastic, you should probably learn the 2nd most sought after skill. Don't wait until you're already looking for a job to try to learn a new technology. The truth is, learning something new will almost certainly help you with your current job as well, so everyone wins.
note: searching hipster-programming-jobs.com to find out what's in demand is a selection bias failure. Use monster.com & dice.com. The goal is to appeal to as many companies as possible, not just the cool ones. You can focus on your dream job once you've got enough experience to ensure that your career doesn't suffer a failure to launch.

Whatever technology you choose will certainly have specific books that you should read; however, there are other books that will also help make up for the experience you (sadly, we) missed by not getting a Computer Science degree.

Refactoring was the first book I ever read that really opened my eyes to better ways to program. It's a pretty easy book to read, even for a beginner, and I can't recommend it enough. It's likely that you find that the most in-demand technology is Java or C#, and Refactoring will complement learning either of those languages. However, if you prefer, there's a Refactoring: Ruby Edition as well.

Once you've read refactoring, it's good to dig into Patterns of Enterprise Application Architecture (PofEAA). PofEAA contains patterns that are found all over the enterprise world (including ActiveRecord). You should read this book not because it is a bible, but because it shows you various solutions to commonly encountered problems. It gives you ideas about how other people are solving problems, and provides reference data that you can refer to if you encounter any of those issues in the future.

Those two books will demonstrate to potential employers that you're interested in building successful business applications; however, you'll also want to read a few books that diversify your experience and open your eyes to what can be done by using less mainstream techniques.

The wizard book and the dragon book sound like interesting fictional books, but are actually building blocks for your programming career. Both books are widely regarded as classics that any serious programmer should read. That said, they are not easy to digest. I'm not going to claim it will be fun, but you should push your way through both of these books - and then do it again a year later. These books can help shape your career in a positive way, especially the wizard book, perhaps more than any other text available.

Once you've pushed through both of those books, there are a few follow up options. If you preferred the wizard book, I would recommend following up with The Joy of Clojure or Learn You a Haskell For Great Good. You're unlikely to get a job using either technology, but both will broaden your horizons and also demonstrate to potential employers that you are deeply interested in increasing your programming capabilities. If you preferred the dragon book, I would move on to Domain Specific Languages. Domain Specific Languages is the definitive guide to DSL programming, and provides you with many examples of how to put DSLs to practical use within your applications.

Once you've made it through all (or most) of those books, you shouldn't have any problem finding a job that is both satisfying and also pays well.

Tuesday, May 22, 2012

sidenum function

sidenum function

The sidenum function appends a sign to a number based on a side. The name sidenum was chosen due to it's similarities with signum in spelling and behavior (dealing with signs).

Definition

The sidenum function of a side x and a number y is defined as follows:

note: 'bid' can be replaced with 'buy', and 'ask' can be replaced with 'sell' or 'offer'. The string representation or type of side varies by trading system.

Usage

The sidenum function appears in most trading applications, though not always named 'sidenum'. It can be helpful for both position related math and conserving screen real-estate by displaying 2 concepts (side & quantity) as 1 (a signed quantity).

Examples

Java
    public static int sidenum(String side, int quantity) {
        if (side.equals("ask"))
            return -quantity;
        return quantity;
    }
Clojure
    (defn sidenum [side quantity]
      (if (= side "ask")
        (- quantity)
        quantity))
Ruby
    def sidenum(side, quantity)
      if side == "ask"
        -quantity
      else
        quantity
      end
    end

Monday, April 09, 2012

Eval a Clojure Snippet in Java

The vast majority of the existing code for my current project is in Java, but the majority of the new code that we write is in Clojure; as a result, I spend quite a bit of time bouncing between Clojure & Java. Recently I was working with some Clojure code that was easily testable (in isolation) within Clojure, but was also executed as part of some higher-level Java integration tests. Within the Clojure tests I used with-redefs to temporarily set the state of a few atoms I cared about; however, setting the same state within the higher-level Java tests turned out to be a bit more of an interesting task.

Our Java integration tests, like all integration tests I've ever run into, look simple enough when looking at the code of the test; however, if you want to understand how the simple test exercises several collaborations you'll need to dig deep into the bowels of the integration test helpers. Integration test helpers are usually a very fragile combination of classes that stub behavior, classes with preloaded fake datasets, and classes that provide event triggering hooks. In my experience you also generally need the High-Level Test Whisperer on hand to answer any questions, and you generally don't want to make any changes without asking for the HLTW's assistance.

Today's experience went exactly as I expect all of my experiences with the functional tests to go:

I added a bit of behavior, and tested everything that I thought was worth testing. With all the new tests passing, I ran the entire suite - 85 errors. The nice thing about errors in functional tests: they generally cause all of the functional tests to break in an area of code that you know you weren't working in. I wasn't that surprised that so many tests had broken, I was adding a bit of code into a core area of our application. However, I wasn't really interested in testing my new behavior via the functional tests, so I quickly looked for the easiest thing I could do to make the functional tests unaware of my change. The solution was simple enough, conceptually, the new code only ran if specific values were set within the atom in my new namespace; therefore, all I needed to do was clear that atom before the functional tests were executed.

Calling Clojure from Java is easy, so I set out to grab the atom from the Clojure and swap! + dissoc the values I cared about. Getting the atom in Java was simple enough, and clojure.lang.Atom has a swap() method, but it takes an IFn. I spent a few minutes looking at passing in a IFn that dissoc'd correctly for me; however, when nothing was painfully obvious I took a step back and considered an easier solution: eval.

As I previously mentioned, this is not only test code, but it's test code that I already expect to take a bit longer to run**. Given that context, eval seemed like an easy choice for solving my issue. The following code isn't pretty, but it got done exactly what I was looking for.
RT.var("clojure.core", "eval").invoke(
  RT.var("clojure.core","read-string").invoke("(swap! my-ns/my-atom dissoc :a-key)"));
I wasn't done yet, as I still needed the HLTW's help on getting everything to play nice together; however, that little Clojure snippet got me 90% of what I needed to get my behavior out of the way of the existing functional tests. I wouldn't recommend doing something like that in prod, but for what I needed it worked perfectly.

** let's not get carried away, the entire suite still runs in 20 secs. That's not crazy fast, but I'm satisfied for now.

Wednesday, December 28, 2011

Clojure & Java Interop

About a year ago I got a phone call asking if I wanted to join another team at DRW. The team supports a (primarily) Java application, but the performance requirements would also allow it to be written in a higher level language. I'd been writing Clojure (basically) full-time at that point - so my response was simple: I'd love to join, but I'm going to want to do future development using Clojure.

A year later we still have plenty of Java, but the vast majority of the new code I add is Clojure. One of the big reasons I'm able to use Clojure so freely is the seamless interop with Java.

Execute Clojure from Java
Calling Clojure from Java is as simple as loading the .clj file and invoking a method from that file. I used the same example years ago, but I'll inline it here for simplicity.
; interop/core.clj
(ns interop.core)

(defn print-string [arg]
  (println arg))

// Java calling code
RT.loadResourceScript("interop/core.clj");
RT.var("interop.core", "print-string").invoke("hello world");
note: examples from this blog entry are available in this git repo. The commit with the code from the previous example is available here and I'm running the example from the command line with:
lein jar && java -cp "interop-1.0.0.jar:lib/*" interop.Example
Execute Java from Clojure
At this point we have Java executing some Clojure code, and we also have Clojure using an object that was created in Java. Even though we're in Clojure we can easily call methods on any Java object.
(ns interop.core)

(defn print-string [arg]
  (println arg "is" (.length arg) "characters long"))
commit

The above code (using the length method of a String instance) produces the following output.
hello world is 11 characters long
Calling a Java method and passing in additional arguments is also easy in Clojure.
(ns interop.core)

(defn print-string [arg]
  (println (.replace arg "hello" "goodbye")))
commit

The above code produces the following output.
goodbye world
There are a few other things to know about calling Java from Clojure. The following examples show how to call static methods, use enums, and use inner classes.
(ns interop.core)

(defn print-string [arg]
  ;;; calling a static method
  (println (String/valueOf true))

  ;;; using an enum
  (println (java.util.concurrent.TimeUnit/SECONDS))

  ;;; using a Java nested (inner) class. Note, in Clojure you
  ;;; use a $ instead of a .
  (println (java.util.AbstractMap$SimpleEntry. "key" "val")))
commit

And, the output:
true
#< SECONDS>
#<SimpleEntry key=val>
Create Java objects in Clojure
When working with Clojure you'll likely want to interact with existing Java objects, but you'll probably also want to create new instances of Java objects. You might have noticed the dot at the end of Abstract$SimpleEntry. in the previous example - that's how you instruct Clojure to create an instance of a Java object. The following example shows the dot notation for calling a constructor of the String class.
(ns interop.core)

(defn print-string [arg]
  (println (String. arg)))
commit

At this point our output is back to the original output.
hello world
When creating Java objects it's often beneficial to know which Java interfaces the Clojure data structures implement. The following examples demonstrate how you can create Java objects while passing Clojure datastructures (and functions) as constructor arguments.
(ns interop.core)

(defn print-string [arg]
  ;;; pass a Clojure vector where Java expects a java.util.Collection
  (println (java.util.HashSet. ["1" "2"]))

  ;;; pass a Clojure map where Java expects a java.util.Map
  (println (java.util.LinkedHashMap. {1 "1" 2 "2"}))

  ;;; pass a Clojure function where Java expects a Runnable
  (println (Thread. (fn [] (println "clojure fns are runnables (and callables)")))))
commit

The output shows the constructed Java objects.
#<HashSet [2, 1]>
#<LinkedHashMap {1=1, 2=2}>
#<Thread Thread[Thread-1,5,main]>
Calling constructors in Clojure is very easy, but that's not always an option when creating a Java object. At times you will likely need to create an instance of a Java interface. Clojure provides both proxy and reify for creating instances of Java interfaces. The following example demonstrates the syntax for using either proxy or reify.
(ns interop.core)

(defn proxy-coll []
  (proxy [java.util.Collection] []
    (add [o]
         (println o)
         true)))

(defn reify-coll []
  (reify java.util.Collection
    (add [this o]
         (println o)
         (println this)
         true)))

(defn main []
  (.add (proxy-coll) "this string is printed on proxied.add")
  (.add (reify-coll) "this string is printed on reified.add"))
commit

note, I also changed Example.java (the details are available in the above linked commit). The syntax for proxy and reify are fairly similar, and both offer additional options that are worth looking into. The primary differences between these two simple examples are:
  • The proxy implementation requires an empty vector where we could specify constructor arguments (if this were an abstract class instead of an interface).
  • The arg list for all methods of reify will specify the reified instance as the first argument. In our example the Collection.add method only takes one argument, but in our reify we also get the instance of the collection.
You might have also noticed that both implementations of add have "true" at the end - in our example we're hard-coding the return value of add to always return true. The following output is the result of running the current example code.
this string is printed on proxied.add
this string is printed on reified.add
#<core$reify_coll$reify__11 interop.core$reify_coll$reify__11@556917ee>
It's worth reading the docs to determine whether you want proxy or reify; however, if you don't see a clear choice I would opt for reify.

Returning objects from Clojure to Java
Our current Example.java returns something from the call to invoke on the clojure.lang.Var that is returned from RT.var("interop.core", "main"), but we're ignoring it so we have no idea what's returned.* Let's change the code and return something on purpose.
// interop/Example.java
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        RT.loadResourceScript("interop/core.clj");
        System.out.println(RT.var("interop.core", "main").invoke());
    }
}

; interop/core.clj
(ns interop.core)

(defn main []
  {:a "1" :b "2"})
Running our changes produces the following output.
{:a "1", :b "2"}
commit

At this point we are back in Java land after making a quick trip to Clojure to get a value. Returning most objects will be pretty straightforward; however, at some point you may want to return a Clojure function. This turns out to be fairly easy as well, since Clojure functions are instances of the IFn interface. The following code demonstrates how to return a Clojure function and call it from within Java.
// interop/Example.java
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        RT.loadResourceScript("interop/core.clj");
        clojure.lang.IFn f = (clojure.lang.IFn) RT.var("interop.core", "main").invoke();
        f.invoke("hello world");
    }
}

// interop/core.clj
(ns interop.core)

(defn main [] println)
commit

The above example returns the println function from interop.core/main and then invokes the println function from within Java. I only chose to pass one argument to invoke; however, the IFn.invoke method has various overrides to allow you to pass several arguments. The above code works, but it can be simplified to the following example.
package interop;

import clojure.lang.RT;

public class Example {
    public static void main(String[] args) throws Exception {
        clojure.lang.IFn f = (clojure.lang.IFn) RT.var("clojure.core", "println");
        f.invoke("hello world");
    }
}
commit

It seems like a fitting end that our final output is the same as our original output.
hello world
*actually, it's the last thing that's returned, or "true" for this specific case.

Thursday, March 24, 2011

Readable Clojure Without a Java Equivalent?

I've recently joined a new team and we've been doing a bit of Clojure. If you've never done Lisp (and I hadn't before I found Clojure) it's natural to ask the quesetion: Will programming in Lisp, especially Prefix Notation, ever feel natural? The question came up a few weeks ago and I had two answers

First of all, I've never been really upset about parenthesis. In fact, I've been doing a bit of Java these days and I don't see much difference between verify(foo).bar(eq(100), any(Baz.class), eq("Cat")) and (-> foo verify (.bar (eq 100) (any Baz) (eq "Cat))). By my count it's the same number of parenthesis. Where they're located moves around a bit, but I don't consider that to be a good or a bad thing.

People also like to bring up examples like this:
(apply merge-with +
(pmap count-lines
(partition-all *batch-size*
(line-seq (reader filename)))))
Stefan Tilkov addressed this in a previous blog entry, and (in the comments of Stefan's entry) Rich Hickey points out that you can use a few different versions if you prefer to lay your code out in a different manner. Below are two alternative solutions Rich provided.
; The same code in a pipelined Clojure style:

(->> (line-seq (reader filename))
(partition-all *batch-size*)
(pmap count-lines)
(apply merge-with +))

; and in a step-wise Clojure style with labeled interim results (a la Adrian’s comment):

(let [lines (line-seq (reader filename))
processed-data (pmap count-lines
(partition-all *batch-size* lines))]
(apply merge-with + processed-data))
I've felt the same way for awhile. You can write Cobol in any language. The real question is, are you up for learning how to solve problems elegantly and idiomatically in a new language. If you're willing to invest the time, you'll be able to find out for yourself if it feels natural to write idiomatic Clojure code.

That was my answer, until today. While working on some Java code I stumbled on the following Java method.
public void onFill(int fillQty) {
this.qty = Math.max(this.qty - fillQty, 0);
}
This is a simple Java method that is decrementing the outstanding quantity state of an order by the amount of the order that just been filled. While reading the line I couldn't help but feel like there should be a more elegant way to express the logic. You want to set the outstanding quantity state to the current outstanding quantity minus what's just been filled, but you also never want the outstanding quantity to go below zero. I read from left to right, and I really wanted a way to express this logic in a way that followed the left to right pattern.

In Clojure, this is easy to do:
(swap! qty #(-> % (- fill-qty) (Math/max 0)))
For readers who are less familiar with Clojure's dispatch reader macro, the above example can also be written as:
(swap! qty (fn [current-qty] (-> current-qty (- fill-qty) (Math/max 0))))
In the example above swap! is setting the qty state with the return value of the function.

If you're really new to Clojure, that might still be too much to take, so we can reduce the example and remove the state setting. Here's the version in Java that ignores setting state.
Math.max(this.qty - fillQty, 0);
The example below is a logical equivalent in Clojure.
(-> qty (- fill-qty) (Math/max 0))
When reading the above Java example I'm forced to put Math.max on my mental stack, evaluate this.qty - fillQty, and then mentally evaluate the method I put on the stack with my new result and the additional args. This isn't rocket science, but it's also not how I naturally read (left to right). On the other hand, when I read the Clojure version I think - take the current quantity, subtract the fill quantity, then take the max of that and zero. The code reads in small, logical chucks that are easy for me to digest.

Obviously, the Java code can also be rewritten in a few other ways. Here's an example of Java code that reads left to right and top to bottom.
public void onFill(int fillQty) {
this.qty -= fillQty
this.qty = Math.max(this.qty, 0);
}
And, I can do something similar in Clojure, if I want.
(swap! qty #(- % fill-qty))
(swap! qty #(Math/max % 0))
While it's possible to write Clojure similar to the above example, it's much more likely that you'd use a let statement if you wanted to break up the two operations.
(defn update-qty [current fill-qty]
(let [total-qty (- current fill-qty)]
(Math/max total-qty 0)))

(swap! qty update-qty fill-qty)
The above example is probably about equivalent to the following Java snippet.
public void onFill(int fillQty) {
int newTotalQty = this.qty - fillQty
this.qty = Math.max(newTotalQty, 0);
}
So, I can write code that is similar to my options in Java, but I'm still left wanting a Java version that is similar to this Clojure example:
(swap! qty #(-> % (- fill-qty) (Math/max 0)))
The only thing that springs to mind is some type of fluent interface that allows me to say this.qty = this.qty.minus(fillQty).maxOfIdentityOrZero(), but I can't think of a realistic way to create that API without quite a bit of infrastructure code (including my own Integer class).

(note, you could extend Integer in a language with open classes, but that's outside the scope of this discussion)

The last Clojure example is definitely the version of the code I would prefer. My preference is based on the way the code reads in concise, logical chunks from left to write. I don't have to solve inside out like the original java version forces me to, and I don't have to split my work across two lines.

I'm sure there are situations where Java allowed me to create an elegant solution that wouldn't have been possible in Clojure. This entry isn't designed to send a "Clojure is better than Java" message. I don't believe that. However, before today I've held the opinion that you can write Clojure that logically breaks up problems in ways very similar to what you do using Java. However, I've now also expanded my opinion to include the fact that in certain situations Clojure can also allow me to solve problems in a way that I find superior to my options within Java.

And, yes, after a bit of time getting used to Lisp syntax, it definitely does feel perfectly natural to me when I'm developing using Clojure.

Wednesday, July 07, 2010

High Level Testing with a High Level Language

In the early days of my project we made the decision to high-level test our Java application with Clojure*. One and a half years later, we're still following that path. It seemed worthwhile to document the progress so far.

My current preferred style of testing is rigorous unit testing and less than a dozen high level tests.

This style of testing doesn't catch everything; however, context is king. In my context, we constantly have to balance the number of bugs against the speed at which we deliver. We could test more, but it would slow down delivery. Since we don't want to drastically impact delivery, we try to get the most we can out of the tests that we do write.

A few more notes on context. Our high level tests are written by programmers for programmers. The application is fairly large and complex. We use Java, Clojure, Ruby, C#, & others. We take advantage of open-source frameworks as well as vendor and in-house frameworks. The result is a user-facing application used exclusively in-house. It's not a service or a 'for-sale' product.

The Bad
Context: The team knows Java fairly well and writes the majority of the domain code in Java. The team didn't have any previous experience with Clojure.
Result: Happiness with using a high level language was often impacted by no knowledge of that high level language.

Context: The vast majority of the code is written in Java and is able to be manipulated with IntelliJ.
Result: Some members of the team felt that using an additional language hampered their ability to use automated refactoring tools, and thus the rewards were not worth the cost. Other members believe the powerful language features provide benefits that out-weigh the costs.

Context: The tests need to run on developer boxes and build machines. One and a half years ago, there was no easy way to run Clojure tests in JUnit.
Result: The team hacked together something custom. It didn't take long to write, but the integration with IntelliJ and JUnit is not nearly as nice as using pure Java.

The Interesting
Context: The team has plenty of experience with Object Oriented (OO) based, C-style languages. The team didn't have any previous experience with Functional Programming (FP). Tests are procedural. However, the team was much more experienced writing procedural code in OO than FP.
Result: The paradigm shift impacted the speed at which the team was able to learn Clojure, but the team was able to peek into the world of FP without writing an entire application using an FP language.

The Good
Context: The team needed to build several utilities that allowed them to high level test the application.
Result: It was trivial to create a REPL that allowed us to communicate at a high level with our application. We practically got a command line interface to our application for free.

Context: The team was curious if Clojure would be the right language for solving other problems on the project. The team knew that with less than a dozen high level tests, rewriting the tests in another language would not be a large time investment.
Result: The team was able to determine that Clojure is a viable language choice for several tasks without having to take the leap on a mission critical component whose size may be variable.

Context: High level tests often require a significant amount of setup code. Clojure provides language features (duck-typing, macros, etc) that allow you to reduce the noise often generated by setup code.
Result: The tests are easier to read and maintain (assuming you understand what and how things are being hidden) and the amount of utility code required to run high level tests was significantly reduced.

*We still use Java to unit test our Java, for now.

Wednesday, June 24, 2009

Java: Method Chain with Snippet

I've noticed a pattern pop up a few times in my Java code in the past 6 months. Maybe it's a decent pattern, or maybe I only have a hammer.

The problem I'm trying to solve is setting some global state, running a test, and ensuring that the global state is set back to it's original (or correct) value. My usual solution to this problem is to remove global state, but not all global states are created equally. The two global states I've been unsuccessful at removing are the current time and system properties.

In my previous post I described how I freeze time using the Joda library. The example code for freezing time was the first time I used a Method Chain with a Snippet. At the time I thought it was an ugly solution, but the best I could come up with.

A few months later I was testing some code that set and read from the system properties. My first tests set the properties and didn't clean up after themselves. This quickly caused trouble, and I found myself turning to Method Chain with Snippet again.

Here's some example code where I verify that setupDir doesn't overwrite a default property:

@Test
public void shouldNotOverrideDir() {
new Temporarily().setProperty("a.dir", "was.preset").when(new Snippet() {{
new Main().setupDir();
assertEquals("was.preset", System.getProperty("a.dir"));
}});
}

And, here's the code for the Temporarily class

public class Temporarily {
private Map properties = new HashMap();

public void when(Snippet snippet) {
for (Map.Entry entry : properties.entrySet()) {
System.setProperty(entry.getKey(), entry.getValue());
}
}

public Temporarily setProperty(String propertyName, String propertyValue) {
if (System.getProperty(propertyName) != null) {
properties.put(propertyName, System.getProperty(propertyName));
}
System.setProperty(propertyName, propertyValue);
return this;
}
}

The code works by setting the desired state for the test, chaining the state cleanup method, and passing the test code as a Snippet to the state cleanup method. The code exploits the fact that Java will execute the first method, then the argument to the chained method, then the chained method.

For the previous example, the 'setProperty' method is executed, then the Snippet is constructed and the initializer is immediately executed, then the 'when' method is executed. The Snippet argument isn't used within the when method; therefore, no state needs to be captured in the Snippet's initializer.

This pattern seems to work well whenever you need to set some state before and after a test runs. However, as I previously mentioned, it's much better if you can simply remove the state dependency from your test.

Wednesday, May 27, 2009

Calling Clojure from Java

Calling Clojure from Java is easy, if you know which classes are important.

On my current project I make all my Clojure files resources, load them, and call the functions directly. The following example shows Clojure printing the argument it's given.

; printer.clj
(ns printer)

(defn print-string [arg]
(println arg))

// Java calling code
RT.loadResourceScript("printer.clj");
RT.var("printer", "print-string").invoke("hello world");

There's a few things worth noting about the example: RT.var takes the namespace name and the function name. The Var returned by RT.var has an invoke method that allows you to pass any number of Objects. The invoke method also returns an Object, which allows you return values from Clojure where necessary.

It's also worth noting that the Clojure/Java interop is very, very good. You could pass a Java object to Clojure, make changes to it in Clojure, and return it back to Java. Of course, you might not even need to return it to Java since the instance referenced in Clojure would be the same instance referenced in Java.

Wednesday, December 31, 2008

Things to dislike about Java

Back in August I started working with Java the majority of the time. I still do a bit of Ruby, but the vast majority of my work these days is in Java. Java is good for a lot of reasons, but that is absolutely not what this post is about. This post is, in no particular order, the things about Java that I really dislike.

Closed Classes
I'm not sure if there's a term for this, but it's the opposite of what Ruby calls Open Classes. Open Classes allow you to define or redefine behavior on any class at any time. Java does not give me this ability. Let's look at a simple example, say I want to define a method that checks to see if a string is null or empty. In a language with Open Classes you can add behavior directly to the String class, so you could check to see if a string is null or empty by asking the string itself.

string.empty?

In an Object Oriented language, where objects are supposed to have appropriate behavior, I like having the ability to add the correct behavior to the correct class.

In Java, I'm stuck doing something very different. I could import some class defined in some library that has way more than I actually need, or, more likely, I'll just define my own StringUtils class and create a static method. In Java 6 I can static import this method, which makes my code procedural, joy.

If you are in the .net camp, don't laugh too loud. Your extension methods are flawed. Until you can override existing behavior, you're only halfway to the solution.

Lack of Closures, Blocks, Anonymous Methods
Whenever I manipulate a list of items I find Java severely lacking. In Clojure I can double each element in an array by simply calling the map method with an anonymous function and a list.

(map #(* % 2) [1 2 3])

In Java I'm stuck with iterating over the array, and adding the results to a temporary collection that is appended to with each iteration. I find that ugly, but it hardly compares to the absolute disgust I have for classes like Comparable and Runnable. An anonymous class instance that has one method is an obvious sign that the language is severely lacking in the area of closures, blocks, or anonymous methods.

Army of NNPPs
Java is currently the "one language to rule them all". This brings about several problems, one of which is the number of terrible programmers writing Java code. I'm lucky enough to work for a company that wants to "build the best software development teams in the world", so I don't work with a single person that writes code I find ugly. Unfortunately, that doesn't save me from the Java NNPP zombie army. Java NNPPs flood the forums and mailing lists with elementary questions and incorrect answers. The amount of signal to noise is so low that I find myself unsubscribing from every public mailing list, and avoiding all Java related forums. It's a shame that I generally have to stick to private mailing lists just to ensure I receive relevant answers.

Literal Arrays & Hashes
Arrays and Hashes are used so frequently, there's really no excuse for not having a literal syntax. In production code this is somewhat painful, but it really shows up in tests. Java does have a few solutions, two of them can be found in the following example.

asList(1, 2, 3, 4)
new Integer[]
{1, 2, 3, 4}

While these do offer "options" they both end up causing problems. The asList method works for about 75% of situations, but leaves me disappointed at least 25% of the time. Far too often there seems to be a method that doesn't like the return value of asList; therefore, I end up doing something significantly uglier to build a "correct" collection. The second example, the traditional Java array also works, but it's ugly to create and often to limited to work effectively with.

Command Line Execution for Snippets of Code
Sometimes you need to do something quick and simple. A command line interface (Ruby's IRB, Clojure's REPL) makes you more effective by seconds or minutes, millions of times a year. That wasted time adds up.

Java (the programming language) is so amazingly verbose (I'm talking about Java, the programming language)
Java forces me to repeat things all the time.

public class NewOrderHandler {
public void onNewOrder(NewOrder newOrder) {
// handle new order
}
}

In 2 lines of code I managed to type "NewOrder" 4 times. Even with a fancy IDE this gets very tedious. When writing Java code I don't actually write anything, I just keep hitting [Ctrl + space] and [Alt + Enter] all day long. When I'm writing code this tends to not be such a problem. I use IntelliJ for all my Java programming and I'm comfortable admitting that I can write Java code as fast as I was writing Ruby code.

The problem is, I don't spend all my time writing code. I spend a fair amount of my time reading code, code someone else on my team wrote, or code I wrote a few months ago. While I can write Java (or at least [Ctrl + Space] Java) as fast as I can write Ruby, it takes me at least 3 times as long to digest the responsibilities of a class written in Java as compared to Ruby.

These days I spend far more time trying to digest a class that takes 4 other classes as dependencies and each of those classes combine with the original class to solve one problem. Unfortunately it takes so much code to get something done that the simple behavior had to be split among 5 classes. To make matters worse everything has to be injected via the constructor because it's the only reasonable way to write testable software. So I end up reading pages of code to understand how each piece collaborates to do one simple task, awesome.

Extensive Tool Support Raises the Barrier of Entry for Tool Builders
I find the Java testing frameworks, much like Java itself, to be a generation behind. There have been steps forward elsewhere and the Java testing frameworks haven't yet caught up. I'm not one for complaining without providing a solution, so I started toying with building a testing framework. Does it integrate with IntelliJ? Of course not. Does it integrate with Eclipse? Absolutely not. Would anyone use it without integration? Emphatic No.

So now it's not good enough to create a good library, now I have to learn about IDEs also. Or (in my case anyway), drop the idea for more interesting pursuits, like finding the language I can replace Java with.

Conclusion
Java's not all bad. In fact, it might be the best tool for the work I'm doing these days. It's definitely tops in the areas that I need to be successful overall. But, I really do miss the things listed here, and all the other things that I've forgotten to write about but consistently swear about at work.

Happy New Year! Here's to a year where I find a suitable replacement for Java.