Trunk Based Development or GitHub Flow

Wait? these aren’t the same? They sure look very familiar!!!

they are very similar but there is one subtle difference, where the release originates

 

 

 

 

 

Using GitHub Flow you release from the branch

 

 

 

 

 

 

 

While with Trunk Based Development you release from master/trunk.

This subtle difference theoretically reduces the chances of regression.

However: if your department is successful in getting the longevity of branches to an absolute minimum: then this discussion will be a lot less important.

 

Source: https://trunkbaseddevelopment.com/alternative-branching-models/

Monolithic Source Control at Google

While you may wonder/think that Google uses git and various fashionable methods interacting with a distributed version control system, they do not!

Here is a great explanation of what Google does regarding source control and more importantly: why they do it this way

Source: https://trunkbaseddevelopment.com/game-changers/

Continuing on this train of thought: What would you use when you want to centralize all the code into 1 repo? Google ditched Perforce over something they developed themselves. While fun and challenging: not an option for us mere mortals.

Perhaps Microsoft comes to the rescue? The ‘New Microsoft’ says they are very pro-open-source and they tried to solve some of the problems with massive git repos by developing GVFS

Introducing GVFS

Soure: https://gvfs.io/ and https://blogs.msdn.microsoft.com/bharry/2017/02/07/more-on-gvfs/

With this implemented essentially you are centralizing the massive repo and only have a ‘sparse repo’ or view on the central repo.

If used right: perhaps this will be beneficial for making a step to Trunk Based Development.

Let’s give this a try!

 

Moving to Google Cloud

After quite a while of procrastinating I finally took the time of setting up my production platform the way I wanted to. In the past my services and sites ran on a VPS that was managed “pet style”. I will sure miss this pet. After 6 years of faithful service it’s time to say goodbye: it’s better this way 🙂
So now all the services are running in a local(ish) datacenter in the north of the Netherlands (brand new!) and I encountered some interesting things in the new product that GCP offers; Google Filestore has a 1 TB minimum which sets the price minimum at a cool $250 dollars a month. Not so cheap when you really only need a fraction of this.
The presentation gives a visual overview of the setup and it’s versions:

  • Version 1: GKE on persistent disks and OnDemand cluster nodes: slow on restart container and expensive
  • Version 2: GKE using Buckets as central store (failed) and Preemptible nodes: not working
  • Version 3: GKE using Google FileStore (beta) and Preemtible nodes: great but expensive
  • Version 4: GKE using homegrown NFS and Preemptible nodes: fast and inexpensive!

POD stuck on terminating?

–force / –grace-timeout=0 does not work?

this worked for me:

This stackoverflow article sent me in the right direction and solved my problem: https://stackoverflow.com/questions/35453792/pods-stuck-at-terminating-status I had to type: kubectl delete pod wordpress-mysql-5f4b68789d-fsxhs --force --grace-period=0 --wait=false
kubectl edit pod wordpress-mysql-5f4b68789d-fsxhs
An editor opens with the manifest for the pod. Remove these two lines:

finalizers:

- foregroundDeletion

Saved the file and leave the editor. The pod with the ‘unknown’ state will be killed immediately and the newly spawned pod will start as usual. Shouldn’t Kubernetes take care about that? I mean: ‘force’ should tell kubernetes that I really really really want to kill a pod. Isn’t it 🙂 ? Josef

Source:  https://github.com/kubernetes/kubernetes/issues/65936

Google Filestore. Managed NFS server, nice but expensive

Even though the price Google asks for it’s new (in beta currently) service is competitive (cheaper than AWS it boasts) the smallest server you can setup is 1TB. Meaning this will set you back a cool ~$240 a month.

while impressive in speed and simplicity of setup; It’s too much for my needs (60Gb) so I’ll just give this a pass for now.

Feature Branching is Evil

From XP-day Ukraine;   A lot of information on Continous Integration and Continuous Delivery. All compatible with what I try to instruct nicely packaged into a 45 min talk. enjoy!

some background on feature branching according to GitHub (the right way) please read on here: https://guides.github.com/introduction/flow/

Event Driven Architecture

Continuous Delivery 3.0! – Integration/Deployment

I was honored to be part of the lecturer team for Continuous Delivery 3.0 in Utrecht; An 8 week training course in which my parts were Continuous Integration and Continuous Deployment.

Enjoy Gitflow, responsibly

Version control was and perhaps always will be a subject of debate. It seems at times we are in a search for a singular “best’ way to manage versions of code under all circumstances. Since about 2008 git has increased popularity significantly and is the most popular version control system since 2012. This meant a new round of debate on how to use the most popular version control system at the time: git. Since it is 2017 now, has the dust settled, and are there any conclusions?

While git is conceptually very different from it’s competitors: it is a distributed version control system and all other older popular version control systems we non-distributed (mercurial excluded). Having no central server is a critical feature when working in a open source setting, for closed source in a corporate environment it is less critical but opens up a lot of new possibilities. Having a distributed or non-distributed version control does not make too much of a difference in the sense of branching policy. With a distributed version control system everyone has a full, authoritative copy of all versions of the code. A central git repository only becomes central because you deem it to be central, rather than the olden days where there was just-the-one server and all others were a client to that central system.

Central questions in usage/policy of software versions were, are and perhaps always will be:

  • When do you branch?
  • What branches are there?
  • When do you merge?

If you standardize on a branching policy there are a couple of options to choose from: Gitflow, Github flow and Gitlab flow. Because of popularity I’ll only discuss the first two: Gitflow and Github flow

Gitflow

First described by Vincent Driessen in 2010 and detailed on his blog describes a straightforward usage of git using a number of standard branches. each of them serving a different purpose:

  • master: Stable software only, everything that is pushed to production. production ready software lives here
  • develop: Integration and ongoing development: things that will be in the next release

Then there are 3 supporting branches for workflow

  • feature: are branches off develop for a “feature development” this is where the main debate is.
  • release: link between master and develop done for integration work
  • hotfix: branch off master for emergency fix

Below is a diagram detailing all branch-types and how the merge back into their originating branch.

The gitflow process is straightforward, well tested and a good way to start. Especially is your current deploy-process is 100% release focussed and has many manual steps in it. There a a couple of downsides, or ‘misbehaviour’ I’ve seen in the wild. Chief amongst which is the policy of starting a feature branch for everything. Step 1 in this antipattern is to create a featurebranch when you start to work on a new item (Bug, Feature, Fix, Improvement). A negative consequence of this is that the branch creator stops communicating with the rest of the world (Team, other teams). It is this pattern that I think is counter-productive to a normal version control policy.

Github-flow

As an alternative to gitflow, the people at github described their version control process. It is clearly based on gitflow but has quite some things removed: gitflow made simpler essentially. Github did this to allows for high frequent releases.

Image result for github flow

As you can see from the diagram it is a simpler branching an merging the idea behind it that you in principle work on the master branch. Only when you suspect features to pose a problem when merging or a speedy stable working using things like feature toggles or branch by abstraction you create a branch for them: a feature branch. When you finish the work you merge the featurebranch back into master.

Going CI

So now that we have the background of version control policies covered it is time to use one of them. I’d suggest starting with analyzing the components and systems first and how they are currently managed in terms of version control. Are there long integration cycles with 1 or 2 releases per year? chances are you are using neither of the two flows described earlier. I bet you use something more traditional where all upcoming releases have their own project-branch.

The change from such a model to a gitflow model is relatively small: you make sure that all development is integrated into the ‘develop’ branch first and make teams work together if needed on the same component. Use external stakeholders to the teams (for instance: architects, leads, ‘component stewards’ ) to play an active role in facilitating concurrent changes on shared components. These people may not be component gatekeepers or judge and jury. This process will take some time in adopting and getting all teams involved in accepting this new way of work. Our tendency as engineers is to be a more of component dictators rather than component stewards. Some engineers go through lengths defending that they, and only they are allowed to perform a change on a component.  This is sometimes even codified in branching policies. The long term goal is not to make all engineers able to do all changes all the time: we need to make sure that the policies do not cause friction in delivering changes to production. Creating a dictator-based bottleneck is a form of friction that we need to remove ASAP.

So now we have a simple, version control policy and are changing the way we deliver software. The next step is to remove more and more of the feature-branches as they are likely to be created by default by all engineers in all teams as soon as you implement any item from the backlog. The reason for trying to remove the mass-creation of feature-branches lies in the communication breakdown between people and teams. The breakdown in communication is always there when you branch: every form of branching diminishes the communication between parts of the organization. This can be as simple as the breakdown between 2 people working in the same team on different features. When you branch you create an isolated world for you or your team to influence: it allows for you to not coordinate with other people who may be working on the same code. If you do this too long (longer than 1 day) you run the risk of missed coordination. This missed coordination can result in merge conflicts but more importantly is a missed opportunity to learn from you colleagues. With some communication and coordination you may have had the opportunity to refactor the code or to design it in such a way that both features are implemented.

When you compare gitflow to github-flow you can see that github flow is less bothersome in the branching since it prescribes less branches and is better suited for smaller batches (releases). This way of working is ideal for Continuous Delivery and is in fact a real Continuous Integration strategy. It is not for everyone though. You can make a comparison to the way in which you can implement Lean in a manufacturing process; one of the hallmarks of lean is that you strive to have as little inventory as possible. The worst way you can implement this is by removing all inventory racks in one go. Striving for less stock inventory will mean you have less inventory racks, in time. The same can be said of implementing Continuous Integration. All branching that goes on are a form of stock inventory: if your process is currently very high on inventory (many, many branches) that means that gitflow will be a better initial fit. Only in time: with lots of focus on the architecture, workflow and processes will you be able to (through continuous improvement) lower your stock inventory. You are able to release software in smaller batches and will find that gitflow branching is becoming bothersome. Right before you reach this moment is the time you switch from gitflow to github flow.

Feature-branches may be a necessity because of the situation the software is in. I believe you must strive to remove as many feature-branches as possible. A change you can make right away is to make the creation of a feature branch conditional. If you implement this simple policy you encourage teams to work together constantly on shared components through the use of common branches and delivery. This tension between teams should result in improvements so that multiple changes can take place concurrently on a component. Perhaps the component should not be shared and split up. this all depends on the efforts done by the team in the technical design of the product(s)

Conclusion

In the end: by practicing some restraint in the use of feature-branches with gitflow you remove some of the main risks. These risks are all related delayed communication and will result in merge conflicts, defects, technical debt and optimizations not applicable to today’s situation.  What remains in gitflow are a number of very useful branches to facilitate integration: A release branch for stabilization, a hotfix branch in case of production emergencies. This base will work nicely when your release frequency is about once every 3 months to once every 2 weeks. When you start to move into the realm of continuous delivery and continuous deployment you will find that even with the banning of feature branches the training-wheels that helped you so well (develop, release, hotfix branch) will start to be a bother. This is an ideal moment to start to move towards github flow. By now you probably have shrunk the delivery size of each new deploy: smaller components, better aligned architecture. In my experience there is little truth in the ‘singular best way to version control’. You can summarize it with a common design mantra: “It depends”. Not all software is immediately ready for a low-inventory way of working like in the github-flow. Using gitflow make the transition easy if you do not fall into the “create a feature-banch for absolutely everything” pitfall. therefore gitflow is a great way to start, if you use it “responsibly”

Fun read: Disrupted by Dan Lyons

Dan Lyons shares what happened to him after being let go working as the technology editor at Newsweek. After a short stint at an online tech news platform he decided to join a startup himself. Hubspot: close to his home (his family and Hubspot are both located in Boston).  Once he joined, he was very surprised that he had in fact joined some sort of dysfunctional cult.  Disrupted describes his ‘misadventures’ in a comical way and makes good points on topics of age discrimination in tech startups; how tech investors have successfully de-risked their investment in the startup company, making the founders and themselves rich(er) while having little to share with the people who happend to work there while the company grew.

A very good read!

disrupted

buy here 🙂

 

 

Crash Course Scrum: Introduction

Last April 22nd I was happy to give a 1 day crash course scrum together with Hugo Messer of Ekipa.co. First time in this setup with co-trainer Hugo.The course consisted of a theory-part in the morning and practical-part in the afternoon.

The practical part of the training is a an exercise in city building using legos. The teams each construct lego buildings for a city-planner (Product Owner). The teams estimated a complete backlog, used 3 sprints and held retrospectives. A near complete scrum experience using Lego

Amazon’s Container Service

The idea behind amazon’s container service is that you focus on providing the containers (Docker images) and amazon defines the rest: datacenter, network, the hosts and auto scale features. Should be interesting if you wish to setup a point of presence in a new geographic location based off amazon’s infrastructure in a few clicks.

Here the introduction video from Amazon (promo) with an example setup

Quick Tip: Ctrl + Alt + & Jetbrains products

Normally when you press CTRL + ALT + <Arrow> in a Jebrains products, this will make you “go back” or “go forward” and jump through the sourcecode. Intel however has been having a different opinion about this since 2008.

If you use Intel graphics card (very popular chip) on Windows and do not reconfigure these “Intel Hotkeys” you will end up with a sideways desktop….. ahh! (to get the desktop back to normal press ctrl-alt UP)

Here is how to resolve this, here is how to disable these hotkeys

  • Search for “Intel” and click the control panel

controlpanel

  • The switch of these “hot keys”

switchoff

(sorry, screenshot in Dutch)

Happy Hacking!

 

Cloud based ZEN Delivery Pipeline

Based on Amazon’s Cloud services I plan to demonstrate a delivery pipeline based on containers using  an online webform for continuous delivery maturity as a test.

High Level Design

ZENdeliveryPipeline

Tools

Area
Tool
Rationale
Version Control Git Popular Open Source distributed version control
Knowledge/Documentation Confluence (10 user) Easy to use, easy to operate, not expensive, using the Gliffy diagram plugin
Continuous Integration Jenkins Most popular build platform
Configuration Management Puppet Use agent to make scale based on ‘contracts’ possible; use health information from agents for monitoring
Binaries Artifactory (OSS)  Allows you to store both Maven based artifact as well as NodeJs based and Docker image layers
App server Tomcat  When building Java web apps, this is a good bet.
Analysis ELK Elastisearch, Logstash, Kibana
Lanugages Java 8  Standard
Build tool Maven  Standard
Hosting Amazon Web Services  Standard
Backup AWS S3 Storage Not too much to backup, AWS glacier not needed
Monitoring Nagios/Puppet  Use Puppet feature and add some Nagios specifics