My Kraków adventure

Having built otto.de in java, modern programming languages that also target the jvm are always interesting for us. We already have Scala in use and are quite satisfied with it. Given that many aspects of a web application are functional in nature, I am sure, a language like clojure will be the best tool for quite a few of our future challenges.

So here is what I learned from all the great talks on day one. The follow up of day two will follow shortly. I will update this post, when videos or slides are published. If you need more detail, you should have a look at the amazingly detailed notes by Philip Potter.

The starter was a talk by Fergal Byrne He told us about Clortex A Machine intelligence software which he is working on and which tries to as closely as possible resemble the structure of the human brain's neocortex. The general strategy is to first study the brain and then build a machine learing software based on it. Thus he started his talk with some interesting insight in how the neocortex in humans and other mammals is structured. There are several, hierarchical layers of neurons. Each layer works roughly the same. Neurons aggregate signals from hierarchically lower layers and their output is aggregated again in the higher layers. Thus the neocortex works with sequences of sequences of neural activation patterns. If you know a little clojure, you might guess that it really shines in mapping such a problem to code. Fergal's work is based on practical and theoretical groundwork by Jeff Hawkins of Palm Pilot fame) who released NuPic a software with the same intentions but built with C++ and python as open source in 2013. While the talk was on a super interesting general topic, there where also some interesting learnings on software development. As a developer I found it interesting how Fergal strictly classifies every part of his software as either core or integration With core being as minimal as possible. Fergal is currently working on a book on machine learning which you can find on leanpub.

Next came a talk by Logan Campbell who shared his experience of introducing clojure at the Australia Post Digital MailBox. 1 million lines of java code, maintenance outsourced offshore, were to be replaced. An interesting part was how he convinced his fellow developers that clojure could be a better solution than scala. One, supposedly quite common, objection was the lack of a type system. It is indeed the case that clojure does not come with type checking, but  this is also a perfect example of the power of a lisp like clojure. While other languages have to be built around typing as a core feature, not so clojure. The core.typed library adds typing support if and only if you need it. Logen showed different implementations for asynchronous request handling that he evaluated and he carved out what their syntactical and functional differences are. In the end Logan gave one very practical tip: If you want to introduce clojure in your organization, do not go for the big bang release. Go live with something small  first. It will be much simpler  to solve problems and concerns on a smaller scale first.

Tommy Hall's talk on escaping the DSL hell by using parenthesis (i.e. clojure) all the way down was equally enlightening and hilarious. It started with a little shock for me, as I was not aware, just how hard puppet fails with namespacing. By this examples and that of other configuration management tools, Tommy demonstrated, that it is usually not the best of ideas to invent your own language. You might a) not really need to and b) it will likely be rubbish. So why don't you c) look into clojure first. Tommy  then discussed Geomlab which is aimed to teach functional programming to children. It has a syntax roughly similar to that of haskell. It does the job quite ok. But. Once the kids have explored all of Geomlab's features they are stuck. To solve other problems, real world problems even, they have to learn another language. Tommy showed how Geomlab is easily reimplemented in clojurescript. A language which allows the kids to keep investigating and to solve problems out of the DSL's scope. See the demo on cljsfiddle press the little play button). Tommy concluded with how the same principal problem (and solution) apply to many if not all other DSLs.

Mathieu Gauthron showcased jvm-breakglass It lets you easily (by means of an extra maven dependency that is) deploy an nREPL server with your JVM application. After you connected with any off the shelf nREPL client, you can analyze and even modify the state of your application using clojure commands in that repl While that is of course every operations departments' worst nightmare it also is more powerful than most other troubleshooting methods from the classical java toolbox like debugging, jmx or using plain log messages. Among the things you can do to your application is accessing public and private members, listing and rewiring spring beans and many others. I can not imagine using this in production myself, but especially if your software is hard to deploy, jvm-breakglass might be a lifesaver one day.

Gary Crawford began his talk on sentiment analysis of the twittershpere with an absorbing recitation from Leiningen vs. the Ants. He then introduced his project on disease prediction using twitter messages as an input. It was interesting to see how much research went into standardised questionaires like PANAS. PANAS-t, an adaption of which can be used to classify the sentiment of the autors of tweets. The core of Gary’s talk was about strategies and techniques to make big data handleable. Tweets are preprocessed and mapped to bitmaps which are then much cheaper to operate on. Geospatial lookup can be done by mapping go coordinates to pixels in a precolored image. Views with different temporal granularity can be efficiently implemented using multiple keys in redis. Two things Gary has not yet achieved: predict the result of the scottish independence referendum and correctly pronounce „Datensparsamkeit“. Anyway, he had a very good point about the latter. 

Next was Paul Ingles who presented on multi armed bandit optimization in clojure. Product optimization cycles are usually to long, complex and slow. Using traditional A/B testing, many organizations do not manage to conduct more than maybe a dozen tests a year. Which is not at all a good number. Using multi armed bandit optimisation the application under development has a number of options (e.g. for the sorting of content). It gets feedback about the success of each of them (e.g. clicks on that content) and presents the best option to the majority of users. All the while it still uses the other options to probe their current success. Paul showed off first the epsilon-greedy algorithm and then Thompson sampling which is in many respects superiour over the former. As in many of the other talks the implementation in clojure was pretty straightforward. Paul even build a little video portal calledNotflixwhich determined both, the sorting of videos and the choice of the best thumbnail using multi armed bandit optimization.