• Clojure/West: Boot can build it

    Video

    Why a new build tool

    • builds are processes, not specifications
    • most tools oriented around configuration instead of programming
    • we’re programmers, we need to be able to program builds

    Our dream build tool

    • made of many indepenent parts that each do one thing well
    • much better than one monolithic program
    • small things are only useful if composition is left to the user

    What is boot?

    • uses maven for dependency resolution
    • we use it to build Clojure, ClojureScript

    A common build workflow

    • no project file required
    • boot -s src show -f - like adding the directory to the path
    • boot -s src -f -- javac -- show -f - compile java, show the fileset

    Installing a jar

    • boot -s src javac -- pom -p boot-demo -v 0.1 -- jar -m boot.Demo -- install - builds and installs jar locally
    • boot -d boot-demo repl - starts a REPL with a dependency

    Same from the REPL

    boot repl
    (set-env! :source-paths #{"src"})
    (boot (javac) (pom :project 'boot-demo :version "0.1") (jar :main 'boot/Demo') (install))
    

    Anatomy of a Task

    (deftask my-task []
        (let [history (atom [])]
            (fn [next-handler]
                (fn [fileset]
                    (swap! history conj fileset)
                    (next-handler fileset)))))
    

    Example

    (deftask build []
        (comp
            (javac)
            (pom :project 'boot-demo :version "0.1")
            (jar :main 'boot/Demo')
            (install)))
    
    (doc build)
    

    Composing your tasks

    (boot (watch) (build))
    

    As a build script

    #!/usr/bin/env boot
    
    (set-env! :source-paths #{"src"})
    
    (deftask build []
        (comp
            (javac)
            (pom :project 'boot-demo :version "0.1")
            (jar :main 'boot/Demo')
            (install)))
    
    (defn -main [& arwgv]
        (boot (build) (show :fileset true)))
    

    As a build.boot file

    (set-env! :source-paths #{"src"})
    
    (deftask build []
        (comp
            (javac)
            (pom :project 'boot-demo :version "0.1")
            (jar :main 'boot/Demo')
            (install)))
    

    FileSet

    • a little anonymous git repo
    • real files underneath but 100% managed
    • basis for Classpath
    • immutable!
    • query API
    • add, remove (which returns a new immutable FileSet)
    • commit: mutates underlying files

    Pods

    • how we avoid dependency hell
    • isolated Clojure runtimes
    • each can have different dependencies
    • easy to create, run code inside of
    • some things can’t be passed between pods

    Example

    boot repl
    (def env (assoc boot.pod/env :dependencies '[[org.clojure/clojure "1.5.1]]))
    (def pod (boot.pod/make-pod env))
    (boot.pod/with-eval-in pod (clojure-version))
    (clojure-version)
    
  • Tailsteak artwork

    I’ve commissioned Tailsteak to create a logo for this blog. All I gave Tailsteak was “clearly a rabbit, and clearly hitech (maybe borg-esque?)”. Here’s are the draft concepts they came up with:

    Draft concepts

    I’m going with the Minority Report-esque version.

    For the blog I’ve asked for a title image “maybe a mad-scientist flavour experimenting in a lab?” Why? Because of the origin of the name for this blog, “Curious Attempt Bunny”. It comes from the german “Neugieriges Versuchskaninchen” which translates to “curious/noisy lab-bunny”.

  • Redeploying Docker containers using GitHub webhooks

    I successfully used python-github-webhooks to rebuild my blog after every push to GitHub.

    I created a push-blog script in the hooks directory in my fork of the repo:

    #!/bin/sh
    
    cd ~/fun/blog
    git fetch origin
    git reset --hard origin/master
    docker run --rm -v "$PWD:/src" grahamc/jekyll build
    

    I setup & ran the webhooks app on my digitalocean box:

    apt-get install python-pip
    git clone git@github.com:curious-attempt-bunny/github-webhooks-docker-deploy.git
    cd github-webhooks-docker-deploy
    apt-get install python-pip
    pip install -r requirements.txt
    python webhooks.py
    

    I pointed my blog repo’s webhook at:

    http://myhost:5000
    

    The effect isn’t instantaneous. It takes around 30-60 seconds for both the github webhook to kick in and then the Jekyll build to complete.

    I’ll use this same pattern for deploying the latest to my other Docker containers.

  • Clojure/West: core.async

    I attended today’s core.async workshop by Luke VanderHart (@levanderhart).

    My main notes:

    Buffers

    There are three main kinds of buffers:

    • fixed buffer (most common) - e.g. (a/chan 42) or: (a/chan (a/buffer 42))
      • blocks when the buffer is full
      • FIFO
    • dropping buffer - e.g. (a/chan (a/dropping-buffer 42))
      • put succeeds when the buffer is full
      • new values are dropped when the buffer is full
      • never blocks
    • sliding buffers - e.g. (a/chan (a/sliding-buffer 42))
      • oldest values are dropped when the buffer is full
      • common usage is a sliding buffer size of 1 (e.g. consuming mouse events)

    Go blocks

    go blocks - eliminate callback hell - an “IOC” process

    • channel operations may “park”
    • parking is cheap (it doesn’t monopolize a thread)
    • will resume when needed
    • go macro rewrites a/<! and a/>! as callbacks - means you can only use them inside a go block (i.e. not extracted to a function called by a go block)

    Alternates

    Can use alternates to timeout reads - e.g.:

    (a/alts!
      [my-channel
       (a/timeout 1000)])
    

    Transducers

    Use Clojure 1.7 for transducers - e.g.:

    (def my-transducer
      (comp
        (filter odd?)
        (map inc)))
    (def buffer-size 42)        
    (a/chan buffer-size my-transducer)
    

    Patterns and Pitfalls

    exception handling: it’s up to you (no mechanisms provided)

    • catch liberally
    • use exception side channels
    • put exception on the channel
    • code defensively

    do use

    • as an abstraction layer between loosely coupled system components
    • to relieve callback hell in an async environment
    • as a primitive in a well-specified abstraction
      • message bus
      • actor model
    • mapping out your channels your diagram should look like a tree or a wheel with spokes (not a crazy complex diagram)

    do not use

    • ad hoc point to point communication
    • FRP (functional reactive programming)
    • as a high-level abstraction
  • Switch to Jekyll

    I was using blogger. I just switched to using Jekyll. I used blogger2jekyll rather than the jekyll import offering because it did a better job (caveat: it needed a little help with redacted comments).

    I’m using Docker to serve this site and I found that the grahamc/jekyll Docker image worked out of the box for me. I have a simple launch script that I use:

    docker run --rm -v "$PWD:/src" grahamc/jekyll build
    docker run -d -v "$PWD:/src" -p 4000:4000 grahamc/jekyll serve -H 0.0.0.0
    

    I’m writing this in advance of moving the CNAME over from pointing at blogger to pointing at my digitalocean host. Once I’ve done that and I’m using nginx for the vhost mapping, then I won’t need the -p 4000:4000 and I’ll instead move to a --link format on the nginx side.

    Curious to know how much RAM you need to run Jekyll in Docker? I’m using the New Relic Server Agent on my host and it’s reporting that Jekyll is using a little over 30Mb.

  • Clojure/West: ClojureScript

    Video

    What’s changed

    • code base has grown
    • speed has improved (even fast with leiningen)
      • analysis caching
      • avoiding analysis altogether
      • AOTed cljs.core (including analysis, source map)
      • AOTed compuled (eventally can get from Maven)

    cljs.jar

    • just need Java 8 (without lein or boot)
      • better for beinngers, fewer moving parts
      • better for the broader ecosystem

    :foreign-libs

    • can include libraries that aren’t written in ClojureScript
    • escape from externs hell
      • allows the community to package useful foreigh libraries
    • CLJSJS now has 30+ popular packages

    cljs.test

    • is official and works with every compiler
    • a port of clojure.test
      • works with :optimizations: none

    Static Vars

    • a subset of Var abstract is available

    macro usage

    • simpler now

    Incremental

    • it used to be easy to corrupt your build
    • child namespaces are recompiled now
    • works even for cold builds
    • for fans of figwheel: :recompile_dependents lets you disable it

    :modules

    • code splits available (more powerful than what’s available for JavaScript)

    conditional reading

    • official support for conditional reading by .cljc extension (works for ClojureCLR, Clojure, and ReplScript)
    • also works in REPLs

    New REPLs

    • JDK 8 Nashorn REPL - almost as has as V8 & JavaScriptCore
    • node.js REPL
      • fastest option and reflective of browser JavaScript performance
    • all REPLs support stacktrace mapping via source maps

    Ambly

    • lets you write ClojureScript React apps to target web apps, iOS devices, and Android devices

    What’s next for ClojureScript

    • API namespaces to hide the internals of ClojureScript
    • optional bootstrap (remove the dependency on Java)
    • ES2015
    • Externs inference (yes, compute your externs for you)
  • Managing multiple ssh keys with heroku

    As soon as you mix personal and work heroku accounts you're likely to run into this error:
    $ git push heroku master

    ! Your account you@yourplace.com does not have access to your_repo.
    !
    ! SSH Key Fingerprint: xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx

    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    Thankfully the heroku-accounts plugin can help you manage your various heroku accounts. Read more about it here.
  • Converting a PDF to PS without the LZWDecode filter

    I wanted to alter diagrams in a PDF and figured I'd convert it to a postscript file to do that. What got in my way was the LZWDecode filter in my postscript file. Here's what I did to get around it:

    1. use qpdf with the --stream-data=uncompress option
    2. use pdftops (not pdf2ps)

    On the Mac I use homebrew to install the qpdf and poppler packages respectively.

    You can convert pdf and ps files using imagemagick, although don't forget to install gs beforehand. The secret sauce option to get high resolution images out was the -density flag. E.g.:

    convert -density 525 some_pdf_or_ps_file page%05d.png

    I hope that helps you!

subscribe via RSS