Getting to use Clojure at work


#1

Hi All,

I thought it might be worthwhile relating my recent success with using clojure at work.

When learning a new language, I find there are two main challenges - learning the language and gaining real-world experience using it. Typically, we can go a long way with personal projects in our spare time. However, this tends to be slow and because it is usually somewhat contrived and based on small or ‘play’ projects, can miss some of the challenges using a language throws at you in the real world.

Convincing management to use clojure in a new project can also be vary difficult. Management is often conservative and needs to worry about things like the number of staff with the skills to support any project based on that new hot technology, avoid becoming dependent on one or two key staff members and when it is something new, how easily it will be to bring in new or additional resources etc.

Clojure has the advantage of being built on top of the JVM, a technology which is vary pervasive in many enterprise situations, but getting permisison to use it for production purposes is still vary hard. However, there are opportunities which arise in the work space where you may be able to get a foot in the door with Clojure. I’ve recently had some success along these lines and thought it might be worthwhile sharing my experiences.

We all know that Clojure and ClojureScript provide some wonderful opportunities with respect to web app development. However, we should also consider Clojure as a pretty good admin and exploration tool which can be used for tasks usually reserved for scripting languages like python or perl etc.

I am currently working on a large Identity and Access Management (IAM) project which is replacing a legacy system with Microsoft Forefront identity Manager. In vary simplistic terms IAM is about the creation and management of digital identities and access to digital resources. This involves extracting data from key source systems (such as an HR system), applying various business rules to create a unique digital identity (i.e. username) and managing access to resources. A big part of it involves managing attributes in directory services like LDAP and Active Directory to control access permissions, user roles and access to various services ranging from email to files and directories on various servers as well as numerous local and remote web applications. In this project, we are dealing with the management of just under 200,000 identities in an environment which consists of Windows, Linux and OSX environments, so tools which are platform agnostic are vary useful (i.e. JVM).

One of the big challenges in such a project is that minor changes in various business rules can have significant flow on impact and you need to deal with a lot of data and analysis of change across multiple directory services and systems. You have to analyze a lot of data and make lots of data comparisons.

This project has been keeping me so busy, I had to put my Clojure projects on the back burner. Then I had a moment of enlightenment. I realised that having a tool with an interactive repl would make my data exploration and verification much easier. I was a little nervous about trying out Clojure - especially as I have to deal with things like Oracle and MS SQL databases, Active Directory and other proprietary systems. There tends to be good support for many open source systems with Clojure, but support for larger systems can be difficult if you need to avoid paying high licensing fees. So I decided to just start off small and see how easy it would be to write some clojure scripts to query and analyse openLJDAP and Oracle databases.

The results were better than I could hope for. I was able to write up a number of simple scripts which made it easy to query and interrogate these systems. I then moved on to Active Directory and a few web APIs. I then created jar file verisons of my tools and provided them to other members of the team, who have found them extremely useful. Most of the ‘scripts’ are fairly trivial and use the tools.cli to provide command line interfaces, though most people are now just running them in a repl because they love the ability to call individual functions in the programs to facilitate various tests and data exploration processes.

This has had two great benefits for me. Firstly, I’ve been able to use Clojure in a real world context, which is just fantastic. I actually look forward to going to work and am vary much enjoying sitting and writing code to solve problems again. The second great advantage is that many of my co-workers are beginning to get vary interested in this Clojure thing. The place where I work has been using Java Spring for some time and a number of staff have seen how easy it is to do similar taks without all the Spring templates and cumbersome framework infrastructure. I have just shown a few of them my simple web based interface to some of the tools I’ve written which use ClojureScript and they are fairly stunned to say the least.

It is probably still a long road to get Clojure and ClojureScript adopted as an accepted alternative to Java Spring etc. However, I suspect we may well see a number of internal tools being developed using Clojure and ClojureScript and this could be the thin edge of the wedge.

However, the main reason I wanted to post this was to highlight an alternative way to possibly get to use a little Clojure at work. Convincing management to use it for a new project is a big ask. However, slowly introducing it as a useful developer tool has some real benefits and will make everyone a little more familiar with the language and may well start a transition. for me, the biggest benefit has been in applying Clojure to real world problems - the benefit of this cannot be under estimated. It has cemented much of my knowledge and provides me with insight on a daily basis which was difficult to get when just hakcing away in my spare time on personal projects.

The other point I wanted to make is that Clojure and ClojureScript is not just for web applications. You can use it for lots of other things and I’m finding it vary useful when it comes to looking at things like system integration, system tooling and data exploration.

Have fun!


#2

Awesome!

Thanks @theophilusx! This was a great writeup.

Did you get any pushback from other developers or management for choosing Clojure for this internal project?

Rock on!
Eric


#3

Certainly have some pushback. Tends to be of two types. However, before I describe them, it is worth describing the work environment. At the moment, I’m working for a large corporate enterprise. In this environment, software and development of software is not a core business activity. It is done to support core business activities (those which bring in revenue). In basic terms, it is not a development or software shop. For anyone who really enjoys programming, these are possibly the hardest/worst environments to work in and can be vary sole destroying. However, these are in fact the largest employer of programmers, so it is where many of us who consider ourselves programmers will end up.

However, it isn’t all negative. These can be good places to work if your willing to do short to medium term engagements which a specific objective (as opposed to just being one of their permanent commodity coders). There are some good opporutnities in such environments provided they are viewed as short term engagements and not a permanent career position. As this fits with the way employement tends to be trending, this will fit with many of us.

OK, on to the two main areas of pushback and the types involved.

Type 1: Co-workers who are either not terribly invested in what they do. These tend to be those who would rather see themselves as a Business analyst rather than a developer. For many of them, having to write Java and do SQL is just one of the downsides of the job. For them, the notion of an ‘alien’ language like Clojure which has all those ‘new fangled’ ideas associated wiht this weird thing called functional programming is just a ‘bridge too far’. There isn’t much you can do about these types. Eventually, they will be replaced by younger programmers and possibly by younger programmers who like to code and are actaully interested in their craft.

Type 2. This type is mainly made up of supervisors and managers. They will tend to push back on anything new. For them, new is a higher risk. It might not work, they might not be able to get people with the right skills to maintain what is developed and there is always the risk that what someone is proposing isn’t the next great thing, it is simply the next great trend and it will die in the tech graveyard of good ideas when never made it (right next door to the betamax PVR graveyard.

The good thing about type 2 is that they can be convinced once they see the real benefits. This is the group which you can demonstrate the benefits to and present good solid arguments with a chance of being heard and being allowed to dip a toe in the water.

So far, I have acceptance to use Clojure for non-production work. That is, I’m using it to provide useful tools and other utilities which, should they not work or somehow fail, will not be a disaster - may be inconvenient, but not much worse. However, as others begin to use these tools and begin to customise them, they also get use to the concepts and even get to like some of what can be done. This can help with adoption.

There is also another type, type 3, who have not done any pushback. However, this is because they just don’t know. I’ve given them jar files representing little utilities and apps they can use to get their job done and they love it. They think it is Java and written in Java. I’m not going to tell them if you don’t.