Dave Snowden: Embrace Complexity, Scale Agility

Fantastic presentation of theoretical underpinnings of a lot of common sense. Dave talks substantially about Cynefin. For more of this, I suggest a much better summary than I’d ever be able to do at this recent The Morning Paper issue.

Some notes I made about complex systems:

  • Everything is about flow; is not about static assessments.
  • Ordered systems are very constrained: one cannot escape the constraints
    • only works in highly ritualized environments
    • a ritual entry into a highly role-based process means you will follow rules you’d never follow as an individual person
    • ritual entry changes the cognitive patterns in the brain
    • order has value, but one has to realize the context; applying it where it shouldn’t be applied is dangerous
    • the obsession with order, making things seem superficially ordered, means the informal network need to work beneath the surface to make apparently efficient systems effective
    • overfocus on efficiency and processes produces perverse results
  • Chaotic systems are unconstrained: everything is operated independently to everything else
    • it’s a transitionary state: whenever there is a crisis people immediately start imposing constraints
    • used deliberately can be powerful: by removing all constraints, innovations can appear, but it’s hard to keep order/constraints from appearing
  • Complex adaptive systems: constraints modify behaviour of participants, but behaviours modify constraints; behaviour and constraints co-evolve
    • everything is in flow and one cannot go back
    • complexity is about inherent and continuous uncertainty; it can be managed as flow, it can’t be managed as static system
    • complex systems are not causal: some things just are; there’s not linear material causality; defined results cannot be predicted; you manage the evolutionary potential of the present moment in time and adjust as you go.
    • flexible, negotiable boundaries are valuable, as rigid boundaries have the habit of becoming brittle and break catastrophically
    • if you don’t change language you don’t change how people think

Production – Designing for Testability

Product – Design for Testability from InfoQ.

Nice talk sharing the experiences that led to and support testing in production as done in Gilt and now at flow.io from Michael Bryzek.

Some notes:

  • Test in production is cheaper: up to almost half your engineering budget could be going to staging environments.
  • Make sure no one else can break your code. If your component fails, it’s your responsibility and no one else.
  • Event streams change failure modes from an unavailability to a delay.
  • Every system and team other than yours is a third party. The team across the corridor is as external to your as Facebook would be.
  • Support data and accounts to be marked as sandbox.
  • Examples in documentation are run against production

Inside the Python’s GIL

This is a slightly dated (2009) but very informative talk from David Beazley about threading in Python and the Global Interpreter Lock.

Some topics include:

  • details of how threads are implemented in python and where the GIL arises from and how it works. I found this very enlightening, specially given how little details the documentation gives about it.
  • signal handling and pathological aspects of its relation to running threads.
  • python’s reliance on the OS to deal with thread scheduling.

Very interesting.

Multiple Perspectives On Technical Problems and Solutions

https://www.kitchensoap.com/2017/08/12/multiple-perspectives-on-technical-problems-and-solutions/

A very interesting articles describing how architecture reviews were introduced and evolved at Etsy.

So what was this architecture review meeting all about? It was pretty simple, really. It was just a presentation by the engineer(s) looking for feedback on a solution they came up with […] The intent here was to help the engineers describe their problem as thoroughly as they can, and by asking questions, we could help draw out any tacit assumptions they made in thinking through the problem.

I really liked to be reminded of Billy Vaughn Koen’s “Engineering Method”, which basically is

“the strategy for causing the best change in a poorly understood or uncertain situation within the available resources”

I’ll probably get back to it in a future opportunity.