Sharing our knowledge

, a 4-minute piece by Dev Mukherjee Dev Mukherjee

Over a period of five years, we have developed a process of software design and engineering that allows a small team to deliver extremely large and mission critical projects. Part of the magic is sharing the core knowledge amongst the team. At an engineering level we achieved it partly by documentation, mostly with maintaining a sharable and reusable repositories of code. We go as far open sourcing our core engineering knowledge, letting everyone benefit from our collective experiences.

Around this time last year we issued beta invites for Twine. The unexpected departure of the person, who had been leading design on our product for several years had us hamstrung and we had to delay the launch. Asides from highlighting obvious issues with our human resources policies, we were witnessing five years of knowledge just walking out of the door (this was also a lesson of the negatives of remote employment, which I will write about at another time). We had been so caught up in pursuing our product future, that we had forgotten to take out an insurance policy on our design knowledge, documentation and education.

On review the above was also partially true of our DevOps knowledge. Being an engineering process this is far easier to capture than something as subjective as design. Sharing knowledge is about sharing the thinking process. It’s role is allowing a team to carry a train of thought. Our biggest lesson learnt was no matter how small or how big a team, sharing knowledge is crucial. People who deliberately protect knowledge are simply threatened by democratisation of information.

Foundation projects

We set out to document our style, thinking, best practices and develop tooling around each portion of interface design and product delivery. Our aim, to create central repositories of knowledge that all of our projects can assimilate from.

These are referred to as our foundation projects. They are offered under the Apache 2.0 license for anyone to use.

Design: We had absolutely no documentation of frontend technologies like Cascading Style Sheets. I am not referring to syntactical knowledge, but best practices and gotchas. Jekyll has been our primarily platform for prototyping interfaces. Every project was setup better than the previous, what we had failed to do was push the newly learnt habits back into older projects.

We started the Jekyll-Foundation project to establish a repeatable process and centrally document good habits of frontend technologies.

Jekyll-Foundation has been maturing over the last number of months with contributions to plugins allowing us to streamline our build process (e.g automatic annotations of style sheets for Google Closure compiler compatibility). It’s not quite mature enough to receive a 1.0 tag but it’s certainly getting there.

DevOps: A parallel project will cover deploying and scaling Python applications on Amazon Web Services. AWS-Foundation will cover security best practices, auto provisioning and scaling of first class citizens of the cloud.

There’s a plethora of out of date information on the subject. Something that all operations engineers have to dig through and keep references to the relevant information.

Our aim is to maintain up to date master documentation complimented by a reference application that can be used to validate the infrastructure.

Epilogue

Alfred Borden calmly put it as “The secret impresses no one. The trick you use it for is everything.” in The Prestige. I share that sentiment, and have decided to document and share all core knowledge that will allow more people to build quality digital products.

Software is Infrastructure

, a 1-minute piece by Dev Mukherjee Dev Mukherjee

Infrastructure are the installations that form the basis for any operation or system. They are often large undertakings, require enormous effort, resources and have unpredictable timelines. Societies rely on infrastructure to function and are taken for granted. Think of the roads you drive on everyday, your telecommunication network, utilities, you expect them to just be there and function.

To achieve such reliability we have put in place stringent standards that infrastructure must adhere to. Unpredictability of timelines and costs are directly proportionate to scale and complexity of the undertaking. Reliability comes not just from the quality of build, but considerations for being able to service the infrastructure while minimising it’s unavailability.

Software is increasingly embedded into our lives, shipping poor software poses a massive threat. To set the basis for a series of articles on great software engineering techniques and the importance of shipping solid digital products, over purely meeting a arbitrary deadline or budget, I would like to redefine what we consider software to be.

Software is infrastructure, and it’s construction and maintenance be treated in the same way as we do with physical infrastructure.

Lucky

, a 2-minute piece by Dev Mukherjee Dev Mukherjee

Transcribed from Episode 1 of My Next Guest Needs No Introduction, David Letterman interviewing President Barrack Obama. His first interview since he left office.

Transcribed verbatim

Barack Obama

But here’s one question I did have in, Cause we’re wrapping up. Don’t you say to yourself boy am I lucky!

And one of the things that I think, I always am surprised by is when I see people who have been successful — in business, or entertainment, or politics — and they are absolutely convinced that it’s all because they are so smart.

And I’m always saying, “Well, look, I worked hard and I’ve got some talent!” — but there are a lot of hard working talented people out there.

There was this element of chance to it. There was this element of serendipity.

And I wonder if you feel that sometimes, and the reason that for me at least, is important — is so A, I don’t feel too self-important but B, you know, you want to see if can maybe figure out how to sprinkle that stardust on other people.

David Letterman

Okay, Mr President!

This is what I have been struggling with at this point in my life.

I have been been nothing but lucky.

When John Lewis and his friends in April of ‘65, March of ‘65 were marching across that bridge, in April of ‘65 me and my friends were driving to Florida to get on a cruise ship to go to the Bahamas because there was no age limit to purchase alcohol, and we spent the entire week — pardon my French, shit-faced.

Barack Obama

Right.

David Letterman

Why wasn’t I in Alabama.

Why was I not aware?

I have been nothing but lucky, and the luck continues here this evening.