• Clojure/West: Simulant in anger

    Goals

    • gather a baseline for performance
    • incrementally improve

    Simulation

    • actions recorded to Datomic

    Validation

    • querying against Datomic
    • you can apply validations to previous versions of your system

    Documentation

    • very few exmples for using Simulant
    • a little bit like Clojure “hard to learn, but it pays you back”

    Findings

    • found functionality that had never worked correctly
    • performance problems
    • faulty assumptions (auto-scaling based on CPU load and not IO load)
    • eroded false confidence

    Summary

    • fantastic for checking the qualities of your system
    • consider simulation testing if your system is high risk [and there’s no good way to drive load to it]
  • Clojure/West: Sessions I missed

    I missed quite a few sessions. Here’s what I heard about them in case you’re considering watching the videos.

    Programming Clojure, Smalltalk-Style

    Appealingly elegant bootstrapping idea. More of a demo. Not much in the way of technical details.

    HoneySQL: SQL Queries as Clojure Data Structures

    Nothing surprising with this talk.

    Data Science in Clojure

    Good to give you an idea of the landscape. Pretty solid speaker.

    Design and Prototype a Language In Clojure

    People accidentally create programming languages so maybe you should take the time to do it properly.

    Composable Healthcare

    Not so much about Clojure and the technology.

    Games and 3D Graphics in Arcadia

    Pretty cool. Still pretty fresh. Probably ready for prime time in a year.

    Creating Beautiful Spreadsheets With Data and Templates

    Pretty good.

    Generating Art In Many Worlds

    Very cool. Introduced a mathematical concept and turned it into imagery, and kept expanding.

    Exploring Programming Clojure In Other Human Languages

    Adapting Clojure to an Intro CS Classroom

    Joys and Pains to Write a Clojure Curriculum for Beginners

    Debugging Clojure Code With Cursive

    Purely Random

    Life of a Clojure Expression

    Domain Specific Type Systems

    Pattern Matching in Clojure: Best Practices

  • Clojure/West: ReactJS landscape

    I attended Luke VanderHart’s (@levanderhart) ReactJS landscape talk.

    Video

    Reactjs

    • your code renders virtual DOM
    • react DOM diffing (reconcilation) commits it to the real DOM efficiently
    • the real DOM generates events that are received by your code, etc.

    Virtual DOM

    • React element - element node in the Virtual DOM
    • React component - custom elements

    Components

    • you write a component specification
    • instances are react elements
    • give us encapsulation and control
    • have properties (set at construction time) that specify look and behaviour
    • have state (updated by the component itself)
    • has a render method and is responsbile for rendering it’s own children
    • has a shouldComponentUpdate method that looks at properties and state and calculates whether the changes need to do anything
    • has lifecycle hooks

    ReactComponent.render() vs React.render(element,container)

    ReactComponent.render() - renders the virtual DOM React.render(element,container) - apply the virtual DOM to the real DOM

    ClojureScipt + React

    • can be much more performant because equality checks on immutable objects are fast in ClojureScript
    • f(data) -> view - React lets you think about the world this way

    Om vs Reagent vs Quiescent

    See Quantum Tic Tac Toe reference implementations.

    Om

    • you want to stay clore to React’s component model
    • you want the full power of React
    • you want immutable data
    • you want the good parts of OO
      • encapsulation
      • modulatiry
      • reuse

    Cursors

    • how Om manages it’s underlying state (via an atom)
    • used to update the application state
    • local access to application state (for encapsulation)
    • behave as values in render context

    Application State vs Component State

    • you have both
    • application state is global to your application
    • component state directly maps to React’s concept of component local state (for local transient things)
    • all updates to application state are via a cursor. E.g. (om/transact! my-cursor the-new-value) - similar to swap.

    Cautions

    • use it only for front-end rendering (don’t write controllers and models as Om components)
    • keep components simple (don’t use lifecycle methods or object local state unless you really have to)

    Reagent

    • you want things to be clean and easy (in a good way)
    • you still want the hard things to possible (but within a simpler paradigm)

    Reactive atoms

    • defines it’s own atom
    • they track dereferences
    • the act of dereferences establishes a reactive link
    • you can have as many atoms as you want (atoms can be shared across components or not)

    Cautions

    • don’t use too many atoms - i.e. don’t use lots of top level global atoms (this ends up as being a large collection of global variables)
    • don’t deref your atoms in every component - deref in a component and pass the values to the child component (otherwise you’ll end up rerendering all components)

    Quiescent

    • is written by the present. Disclaimer!
    • you like functional, value-oriented programming
    • you want to do your own app state management
    • component state is bad
    • all rendering is top-down

    Cautions

    • it’s lower level
    • choose good higher-level idioms
    • you may have to abandon it at some point

    Kioo

    • you can use Kioo with Om, Reagent, or Quiescent for HTML template rendering

    My summary

    Reagent looked the most appealing based on the material of the talk.

  • Clojure/West: Parallelism in Clojure

    Video

    Main ideas

    • clojure has good support for parallelism
    • it helps to know how things work
    • there are tools

    Future

    (def my-future (future expression))
    @my-future                          ; dereference the value
    
    • uses a Java solo executor under the hood

    Limitations

    You’ll have problems if

    • you expect exceptions to work normally
    • you want to control the number of concurrent threads
    • your tasks are small

    pmap

    • runs roughly 2 + number of cpus items in futures
    • generates threads as needed
      • beware simultaneous pmaps
      • it’s wacky when there’s chunking
    • a slow task can stall it (due to the implementation)

    Limitations

    • similar to future
    • you need to force evaluation of a pmap because it’s lazy

    core.async

    • uses CSP channels and coroutines (coroutines are lighter weight than threads)
    • reads a lot like Go
    • read like one flow: avoid callback hell
    • uses cooperative multithreading
    • backed by a fixed-size thread pool (48 + number of cpus)
    • best for async programming (not so much parallelism)
      • you shouldn’t block many of its threads
      • easy to wait on other work
      • you might use it to interact with worker threads
    • parallelism using pipeline
    • warning: exceptions can kill coroutines

    Claypoole

    Disclaimer: Claypoole was written by the author.

    • use thread pools to control parallelism
    • can auto-manage thread pools (the JVM will not clean up your thread pools for you)
    • tries to get things done fast
      • it’s eager by default: not for infinite seqs!
      • output is “eagerly streaming sequence”
    • doesn’t stall on slow tasks in the way that pmap does
    • streaming sequences can be chained
    • cp/future, cp/pmap, cp/pfor, cp/pvalues, …
    • unordered functions available - cp/upmap, cp/upfor
    • lazy versions available - cp-lazy/pmap
      • can be better for chaining tasks (no risk of growing buffers between pmaps)
    • exceptions get rethrown correctly
    • eliminates chunking
    • priorities available - cp/priority-threadpool and :priority value

    Reducers

    • r/map, r/fold
    • uses Java’s Fork/Join pool for flexibility
    • useful for cpu-bound operations

    Tesser

    • t/map, t/fold
    • avoids Java’s Fork/Join pool for performance
    • useful for cpu-bound operations
    • distributable on Hadoop

    Summary

    • Clojure has built-in parallelism
      • future: thread from pool
      • pmap: lazy future sequence
      • there are some drawbacks
    • There are tools
      • core.async: for asynchronous & CPU loading
      • claypoole: for thread pools, controlling degree of parallelism
      • reducers: for CPU loading, parallel reduce
      • tesser: for CPU loading, parellel reduce, and Hadoop
  • Clojure/West: Intro to Trapperkeeper

    Video

    What is TrapperKeeper

    • hosting framework for long-running applications and services
    • created by Puppet Labs

    What TrapperKeeper does for Puppet Labs

    • code generalization
    • component reuse
    • manage state
    • lifecycle management
    • dependency management

    Features

    • turn services on and off via configuration
    • multiple web apps on a single web server
    • unified logging and configuration
    • simple configuration syntax
  • Clojure/West: Everything Will Flow

    Queues

    • unbounded queues are fundamentally broken (they are bounded - e.g. by memory)

    Drop data

    • only valid if newwer data obsoletes older data

    Reject data

    • often the only choice for an application

    Pause data (aka backpressure)

    • often the only choice for a closed system, library, or sub-system
    • backpressure isn’t free - it hurts our overall throughput

    This means

    • plan for too much data
    • use backpressure wherever possible
    • buffers give us throughput at the cost of latency - avoid composing buffers

    Learning from the consuming Twitter feed an example

    be clear what “completion” is

    • be picky about your tools
    • prefer metrics to raw performance
    • you’re never measuring all of your system

    Learning from the chat room example

    • manifold tracks the queue dependencies

    #

  • Clojure/West: Composing Interactive Apps With Zelkova

    Zelkova

    • based on Elm
    • a reasonable language for talking about time
    • signals are central

    Signals

    • values transformed by pure functions
    • shared freely
    • synchronous and deterministic
    • think of signals representing values over time

    Example

    Folling the mouse

    (defn steering
        [state mouse-pos]
        (assoc-in state [:flyter :position] mouse-pos))
    
    (def app-signal
        [init-state]
        (z/reductions steering
                      init-state
                      mouse/position)) ; mouse/position is a signal
    

    Steering

    (defn steering
        [state mouse-pos]
        (let [flyer-pos (-> state :flyer :position)
              diff      (v/subtract mouse-pos flyer-pos)
              angle     (v/vector->radians diff)])
        (assoc-in state [:flyter :angle] angle))
    

    Thrust

    (defn inertia
        [state time-delta]
        (let [velocity (-> state :flyer :velocity)
              translation (v/multiply velocity time-delta 0.2)]
            (update-in state
                [:flyer :position]
                (partial v/add translation))))
    
    (defn acceleration
        [state toggle]
        (if toggle
            (let [angle (-> state :flyer :angle)]
                (update-in state
                           [:flyer :velocity]
                           (fn [velocity]
                                (->> angle
                                    (v/radians->vector)
                                    (v/with-magnitude 0.2)
                                    (v/add velocity)
                                    (v/limit-magnitude 1)))))
        state))
    
    (def app-signal
        [init-state]
        (let [time-deltas (time/fps 30)
              inputs      (->> (z/map vector 
                                 time-deltas
                                 keyboard/space
                                 mouse/position)
                                (z/sample-on time-deltas))]
            (z/reductions (fn [state [time-delta spacebar? mouse-pos]]
                            (-> state
                                (inertia time-delta)
                                (acceleration spacebar?)
                                (steering mouse-pos)))
                          init-state
                          mouse/position))
    

    Signal graph with Om

  • Clojure/West: Building CircleCI's frontend with Om

    Video

    The tour

    • no private APIs
      • great for dog-fooding public API
      • not perfect for the UI’s needs, but good enough
    • skip discussion of Compojure, Ring, etc
    • circleci is open source

    Architecture

    • controllers swap into app state, that renders into the view

    The atom

    • the good
      • debuggable
      • simple
      • easy
    • the bad
      • anti-modular - all views must agree on the shape of the data
      • complex - couples API requests to the views
      • hard - discipline required to manage the schema of the atom at scale
    • the ugly
      • fetch vs render performance conflict
      • denormalization is the norm
      • tastes like a shared filesystem

    Controllers

    • page centric
    • centralized
      • blessing - global logging, easily audited
      • curse - no component isolation
    • not-deterministic
      • take network non-determinism with core.async
      • inadvertently introduce UI non-determinism

    Components

    • 180 Om components
    • pure components
    • assume controllers pre-fetch all data
    • minimize component-local state - don’t be afraid to use local state
    • limit callbacks to message enqueues
    • encapsulate violations of the above

    Markup

    • a pain point
    • syntax really matters
    • hiccup works well for them
    • but: react breaks “data”, e.g. print/read
    • want to normalize primitives vs components vs functions

    Asset builds

    • ClojureScript builds
    • wanted: asset pipeline - ClojureScript friendly build on-demand tooling needed

subscribe via RSS