[This is a slightly modified version of an internal travel blog post I wrote for the Vend crew]
After a couple of nights of 4-hour jet lagged sleeps, I caught the F-train subway from my sister’s house (near Prospect Park) to Downtown Brooklyn. Turns out that the inside of the Brooklyn Marriot is the same as any conference centre in the world, regardless of how much Beastie Boys I listen to.
I sat through 7 talks, and will try to summarise my top 3 each day.
TL;DR: We’re doing cutting-edge work in a supportive team culture at Vend. There are some incremental ways we could improve as we grow, but nothing earth-shattering.
Tweet of the day: https://twitter.com/tomtheguvnor/status/476846637130985472
Adrian Cockcroft (basically “Mr Microservices” from Netflix, now working at a VC firm) talked about the culture and process of moving to the cloud and micro services. The talk was more aimed at big enterprises trying to break down monolithic IT departments, but there were some good points:
- Netflix has a policy: If a meeting happened, you should try your best to remove the reason for the meeting in the first place.
- Stop doing any “undifferentiated heavy lifting”. Find a XaaS or supplier to do it for you.
- Disruptors take something that used to be expensive and work out a way to “waste” it. E.g. cloud based SSD storage. Incumbents still treat it as precious.
- Even if you’re doing CI and deploying multiple times a day, QA & Integration is hard at scale.
- Gets really hard in the 100-dev range.
- Maxes out at 10’s of deployments per day.
- This is where microservices come in. They can reduce or remove overlap between teams.
- Doing Microservices properly means no running components are changed. Leave the old ones there and deploy new ones, then slowly route traffic to them. “Waste” more VMs so that you can experiment and roll-back easily.
In general, microservice architecture is getting constant airtime at the conference, but I’ve yet to talk to anyone using it in a seriously transactional environment. Netflix helped pioneer it, but the way I understand it, Netflix is extremely read-only in comparison to Vend, and probably can afford to lose some input data occasionally. We can’t.
Jez Humble (Mr Continuous Delivery, now also Mr Lean Enterprise) talked about Kanban, Jidoka, and continuous improvement. He said “Lean doesn’t mean cutting costs. Lean means investing to reduce waste.” – which lends itself to carefully and analytically eliminating areas of waste and inefficiency (calling back to Adrian’s comment about removing meetings).
He also pointed out that a high trust culture is a predictor of efficiency, and recommended reading about the Nummi car plant, where Toyota re-hired a poorly performing GM workforce and completely turned it around though application of their process and policies. Foremost of which being a no-blame culture, especially during stop-the-line events. I reckon we do an awesome job of this already.
Jez reminded me about Toyota’s “Improvement Kata” (which I think we could and should do better, especially out of Retros).
My third favourite talk was by Edmund Jorgensen, on ways to improve delivery velocity. He had a pretty hilarious allegory for talking about cruft or technical debt (or other causes of slower delivery), and the way that non-technical people see it:
Imagine a stock warehouse with lots of forklifts. They drive around ok most of the time, but occasionally Ninjas appear and muck things up, slowing down the forklifts. The only way to make the Ninjas go away is to do a juggling display for them. Every time we complain about the ninjas, and non-technical users come to see, the ninjas disappear. All they see are juggling forklift operators, so they say “Stop bloody juggling and get back to work!”
His solution: don’t talk about Ninjas when talking about the Ninja fight. Kinda silly, but he did suggest finding alignment between what makes the user’s thing faster or better, and what makes our engineering stuff easier to build and therefore faster to deliver.
I think we already do this very well, by improving our architecture as we build or fix features. And given our required pace, I can’t see us stopping all the forklifts for an all-out Bruce Lee Ninja Fight.
So yeah, a pretty intense day. Tune in tomorrow for Eric Evans, Aish Fenton and more.