QCon NYC Day Two: Skills To Pay The Bills

[This is a slightly modified version of an internal travel blog post I wrote for the Vend crew]

It’s a dreary Thursday in downtown Brooklyn, low cloud, muggy – but at least I got some good sleep. The F-Train was packed this morning.

TL;DR: Embrace imperfection and change. Still no one doing transactional microservices.

Tweet of the day: https://twitter.com/darinrs/status/477191543577509888

The day kicked off with Dianne Marsh, Director of Engineering at Netflix. She didn’t elaborate a lot on what Aidrian said about Netflix culture on Wednesday, but did list through a bunch of tools they use to do their builds and deploys. They use tools like Asgard and Animator to manage and deploy Amazon Machine Images (AMIs) directly. Apparently this is pretty old-school compared to what we’re doing with Puppet; and Chef, and new containerised deploy tools.

Continue reading “QCon NYC Day Two: Skills To Pay The Bills”

QCon NYC Day One: No Sleep Til Brooklyn

[This is a slightly modified version of an internal travel blog post I wrote for the Vend crew]

After a couple of nights of 4-hour jet lagged sleeps, I caught the F-train subway from my sister’s house (near Prospect Park) to Downtown Brooklyn. Turns out that the inside of the Brooklyn Marriot is the same as any conference centre in the world, regardless of how much Beastie Boys I listen to.

I sat through 7 talks, and will try to summarise my top 3 each day.

TL;DR: We’re doing cutting-edge work in a supportive team culture at Vend. There are some incremental ways we could improve as we grow, but nothing earth-shattering.

Tweet of the day: https://twitter.com/tomtheguvnor/status/476846637130985472

Adrian Cockcroft (basically “Mr Microservices” from Netflix, now working at a VC firm) talked about the culture and process of moving to the cloud and micro services. The talk was more aimed at big enterprises trying to break down monolithic IT departments, but there were some good points:

  • Netflix has a policy: If a meeting happened, you should try your best to remove the reason for the meeting in the first place.
  • Stop doing any “undifferentiated heavy lifting”. Find a XaaS or supplier to do it for you.
  • Disruptors take something that used to be expensive and work out a way to “waste” it. E.g. cloud based SSD storage. Incumbents still treat it as precious.
  • Even if you’re doing CI and deploying multiple times a day, QA & Integration is hard at scale.
    • Gets really hard in the 100-dev range.
    • Maxes out at 10’s of deployments per day.
    • This is where microservices come in. They can reduce or remove overlap between teams.
    • Doing Microservices properly means no running components are changed. Leave the old ones there and deploy new ones, then slowly route traffic to them. “Waste” more VMs so that you can experiment and roll-back easily.

In general, microservice architecture is getting constant airtime at the conference, but I’ve yet to talk to anyone using it in a seriously transactional environment. Netflix helped pioneer it, but the way I understand it, Netflix is extremely read-only in comparison to Vend, and probably can afford to lose some input data occasionally. We can’t.

Jez Humble (Mr Continuous Delivery, now also Mr Lean Enterprise) talked about Kanban, Jidoka, and continuous improvement. He said “Lean doesn’t mean cutting costs. Lean means investing to reduce waste.” – which lends itself to carefully and analytically eliminating areas of waste and inefficiency (calling back to Adrian’s comment about removing meetings).

He also pointed out that a high trust culture is a predictor of efficiency, and recommended reading about the Nummi car plant, where Toyota re-hired a poorly performing GM workforce and completely turned it around though application of their process and policies. Foremost of which being a no-blame culture, especially during stop-the-line events. I reckon we do an awesome job of this already.

Jez reminded me about Toyota’s “Improvement Kata” (which I think we could and should do better, especially out of Retros).

My third favourite talk was by Edmund Jorgensen, on ways to improve delivery velocity. He had a pretty hilarious allegory for talking about cruft or technical debt (or other causes of slower delivery), and the way that non-technical people see it:

Imagine a stock warehouse with lots of forklifts. They drive around ok most of the time, but occasionally Ninjas appear and muck things up, slowing down the forklifts. The only way to make the Ninjas go away is to do a juggling display for them. Every time we complain about the ninjas, and non-technical users come to see, the ninjas disappear. All they see are juggling forklift operators, so they say “Stop bloody juggling and get back to work!”

His solution: don’t talk about Ninjas when talking about the Ninja fight. Kinda silly, but he did suggest finding alignment between what makes the user’s thing faster or better, and what makes our engineering stuff easier to build and therefore faster to deliver.

I think we already do this very well, by improving our architecture as we build or fix features. And given our required pace, I can’t see us stopping all the forklifts for an all-out Bruce Lee Ninja Fight.

So yeah, a pretty intense day. Tune in tomorrow for Eric Evans, Aish Fenton and more.

Politics and Social Media

I’m hoping for a change of government in New Zealand come September, but not for the reason you may assume.

Back in 2008 when John Key came to power, Twitter was barely kicking off and Granny Herald had about 40 or 50 thousand more subscribers than today. I genuinely can’t remember being involved in any real-time online conversations about Helen Clark’s Labour government, whereas I now read multiple personal opinions on a daily basis about the latest thing that Key or Brownlee or Cunliffe or Norman has done.

I guess it’s obvious that Trevor Mallard is just as bad as Judith Collins on Twitter, but what about Chris Carter, Cullen and Goff? What would their time in power look like through the lens of personal jibes and dumb hashtag jokes? Did they get a relatively free ride?

When National put through not one, nor two or three, but four laws that appear dodgy in respect of the Human Rights act, I paused before flipping my lid online: surely it can’t be this bad? Surely other governments have done similar things and it’s just my pinko-liberal world-view to blame. Chill out. Right?

Is the Fifth National Government truly a cesspool of dimwitted decision making, or do I only think that because the Twitter echo chamber tells me to do so? I like to think I’m equally critical of Cunliffe’s gaffes and lack of policy, but worry that I’m just suffering confirmation bias. Does John Key truly disagree more with scientific, statistical, and economic consensus than Helen Clark did, or do I just think that because there are more immediate howls of anger from the e-left (iLeft? cyber-left?) when he comes up with another pearler about his “advice” saying otherwise?

To me it looks like no politicians are getting an easy ride online right now, when any fact or opinion can be rapidly fisked by the pyjama army ranks on both the left and right. If anything, it appears that investment in polling and carefully researched talking points is many times more valuable now than it was in the time of controllable media. Just take a look at the nutty “infographics” and crappily crafted motivational posters coming thick and fast from all sides lately: any time a politician says something vaguely quotable it’s turned into a Facebook-shaped meme before bedtime. Most of these are pointless crap, but I guess the plan is if just one of them turns “viral”, then yay votes!

The question I’m asking is this: what does social media look like when your favourite team ends up winning all the time instead of losing? I’m a Greens voter, but I like to think I’ll still be calling bullshit on dumb policy if they’re ever in power. Will they turn out to look just as idiotic as the current bunch? Will Trevor Mallard be an appalling speaker, biased toward the left instead of driving for genuine answers at Question Time?

Basically: bring on a change of government so I can check my bias by trolling left-wing politicians. Also: you’re on notice left-wing echo chamber. I’ll be watching.



We Are Treating it Very Seriously

After reading Lance’s latest post, I remembered an interview I heard on the radio this morning. It was about a serious workplace accident, at a company that has had two recent workplace deaths. The man being interviewed said:

“We are treating this very seriously”

I’m going to come right out and say no, you are not. You are not treating this very seriously at all. Neither are Auckland Transport, the NZTA, nor countless other organisations dealing with the interaction between humans and heavy machinery.

“Treating it very seriously” is not getting on the radio to talk about treating it seriously. Treating it seriously is protecting human life and limb at any cost. That’s all there is to it. Goods, services, transport – what’s the point if we’re killing people in the process?

Shut it down and don’t open it up again until you can guarantee me that no one will die at your port.

Ban cycling from all inner city routes until you can grade-separate cyclists from traffic. Shit, in many cases (Ponsonby Road, Parnell Rise, Tamaki Drive) all you need are some portable barriers and the removal of carparks, and cyclists are now safe. At the cost of what? Some extra walking time for people having to park farther away.

And don’t you dare fucking preach about the inconvenience. Inconvenience? Go and talk to the family of a dead person about inconvenience.

We only have our lives.

Smart TV is Bullshit

Smart TV is a pile of arse. Check out this latest fucktastrophe* from LG:

“When you first turn on the TV, an animated character called Bean Bird appears to help guide you through various options.”

What the shit? LG gets WebOS, lauded as one of the most promising operating systems of recent time, and uses it to create fucking Clippy for television?

It’s ironic that after reading all these “you must create!” missives, my first long-form blog post in god knows how long is inspired by the desire to burn down the creations of others, but stick with me here: “Smart TV” needs to die in a fire.

Standard-issue CD for TV interface designers

For decades now (pretty much since the invention of the remote control), TVs have had on-screen displays, which have been getting more terrible with each passing moment. More menus, more options, more inputs: all artfully designed by some half-blind shitbird with a “250,000 Web ClipArts” CD-ROM.

And somehow, in an age with practically unlimited computing power, TV manufacturers managed to build user interfaces with the responsiveness of a rolled pork roast. What the fuck is up with that? It’s not even like there are space or heat constraints limiting the chips they can use.

All this time computers and phones have been getting more usable and more responsive. What have TV manufacturers been doing? “Why Ben,” you say, “they’ve been adding features!”

Features like an unusably slow, impossible to navigate web browser! A shitty walled-garden tick-the-box-we’ve-shipped-it-boss app store! How about this awesome streaming video service that proxies traffic through our servers in Asia?  And don’t forget Angry Birds!

Fuck. Off.

Just stop. For one second stop and make me a television that looks great, operates quickly, and gets itself the fuck out of the space between me and my video entertainment. Please.

AND: if you feel the need to create a god-damn animated character to help people understand how to set up their TV, step the fuck back and ask yourself WHY you got to that point. Look yourself in the mirror, you “Smart TV” charlatans. Go fix something that is broken for a change.


*Credit to Nat Torkington for the word “fucktastrophe”.