One feedback we heard back often from our users is that they wish they know more about what the product team is working on, what new things are coming and in general, they want to hear more from us. We are creating a new section here called The Builder’s Corner where we will share our thoughts, product roadmap and other important topics.
The first thing I would like to share is our approach to building Versa Cloud ERP. But I first need to go back in time. I used to work for Sage which is one of the biggest software providers for small and medium size businesses around the world. Before web/cloud is popular, almost all of Sage’s software were running on desktop. This means someone has to go install the product from a CD. Any updates have to be installed manually either via a new CD mailed to users or via service pack downloaded from a website. Typical release cycle is one new version per year and 2-3 service packs a year. The product team would spent a year building out a new version and then released the product. Rinse and repeat. Move on to the next release cycle.
Almost all the software in the industry were distributed this way and updates to software were infrequent. Some products have multi-year release cycles but the products I worked on generally are on an annual cycle. Users are trained to wait for an annual release and sit on tight for another year before anything else can be expected. Users were taught by the service team to use workarounds for bugs that will not be fixed for another year. This model is not perfect but worked most of the times.
There are a few key issues with this type of release cycle.
First issue is that product roadmap is planned annually and often contains many major features. A lot of time is spent on planning and scheduling at the beginning of the release cycle. If an existing competitor launches a major product changes or a new competitor comes to town, all that planning either has to be reworked or completely thrown out. Management dealt with this issue by packing feature release in service pack. Service packs are minor release to the software after a major release. It is meant for fixing bugs discovered in the new version after a release. But someone came up with the idea “hey, we can just put this new feature here in service pack 1 and this new feature in service pack 2”.
Second issue is that we were working in a linear (or waterfall) fashion. Development team work on a long list of features. When a feature is completed , it is sent to QA for testing and then development team work on the next feature and QA tests it again. Then at the end of the year, the entire product is tested end to end and what often happens is that QA would discover many features tested earlier in the year no longer works the same way as it was tested or completely different. The reason is that since many features are worked on in a linear fashion, the changes made later in a product development cycle introduced unforeseen issues with the UI, the database schema or workflows. Development team would just make the changes to accommodate the new features. Sometimes, a team member would write down what was changed and communicate to the QA team but often QA team had to discover such deviations on their own. This of course caused friction between the development team and the QA team. QA team had to spend a lot of time towards the end of a product development cycle for regression testing: re-test all the feature from the beginning of the the year till now. This made end of the product development cycle action packed. Everyone was busy fixing things newly discovered and retesting them. I remembered pulling all nighters in order to meet schedule. Teams were often asked to work over time during such times. Management would bring in dinner for everyone. Every year, we scrambled towards the end and pushed really hard to get the product “out of the door”. Management tried to deal with this crunch time by dividing development cycle into multiple stages. We have integration point 1 to integration 3. At each integration point, all the features build before that are supposed to be complete and working together. They will be tested together and entire product is tested end to end. The idea is to have a working product at each integration point. This did reduce the crunch at the end but it did not complete eliminate it.
Fast forward a few years and still at Sage, I worked on a few cloud/web products. The release cycles were no longer annual. Agile development has caught on with Sage and we replaced annual release with quarterly releases. Each release is build using sprints between 2-4 weeks in length. Automated tests were introduced to reduce amount of manual testing. We typically released a new release every 2 to 3 months. Product roadmap was still planned annually with multiple releases.
Then I started working on Versa Cloud ERP and we have a need to update the product often. We have tried a few release schedules. We started with quarterly release, then monthly, then eventually we started doing weekly releases. How we got to the weekly cycle is mostly because of me. I often interact with customers given my role here and I have this bad habit of answering questions from customers on when would this be fixed with “next week”. For some reason, I always think anything can be fixed in a week. But of course certain things take longer than a week. But now I have set the customer’s expectation and we had to release something every week. At first, I though this is suicidal and non sustainable. How can we go through a product release cycle (one that used to take a year) in a week. We have to plan a feature, work on it, test it and then release it. All in a week! We learn to divided and conquer – divide features into small chunk so it can be worked on in a week. Big features would still take months to develop but we still plan the work for big features weekly. We also started to invest on our automated test suites which ensure verify various features still function as intended in the product. It is impossible to manually test a product end to end every week so we have to rely on automation. There are certain things automated tests cannot catch. Like the button that used to be on the right is now on the left. So we still have to do some manual QA. After almost 2 years of doing this, I would say we got into a rhythm of doing weekly releases but of course we had made mistakes along the way and broke things on our weekly releases. Certainly, we have to be better at catching things before deployment. But due the weekly releases, it means major problems would take at most one week to correct. Comparing to what I used to do, this is light speed.
Now a few days ago I read about a company deploying multiple times in a day. Maybe a daily deployment is the next thing to try? Some food for thought.
