Dealing With Large Features in Git Repos

By


There is not a "correct" way to organize your servers and git repos and each company has its own problem and solutions. Some companies use test servers to run some very basic verifications in their codebase before deploying new features to production. Some even have a staging server that is as similar to production as possible, where new features are tested for a last time before deploying to prod. Those servers usually run deployed code that is in the development, staging, and master/main branches in the git repo.

This would be what the usual flow of development looks like:


Another good practice many companies adopt is the versioning of the code to keep track of what is deployed on each server. There is a lot about versioning alone that should be discussed in other posts, but let us adopt the format major.minor.build for our example.

Dealing with Bugs in Tests


Now, let us imagine that a bug was discovered in one of your features when it was being tested in the Test Server. We do not want to merge any code to Staging that has a known bug, but then, how do we deal with it if more features are also being deployed to Test?

The first option we have is to just work on a fix for our buggy feature and deploy all other features once it is fixed:


This is nice because each version number on all servers has the same code, it is easy to merge from development --> staging, and from staging --> master. The only bad part: we have to wait for the fix to Feat. 1 to be ready before we can merge. Now, what happens if Feat.2 and Feat.3 also have bugs discovered? Yes, everyone gets blocked.

One alternative to this problem is to revert the change from Feat.1 as soon as you find it.


This works well because now Feat.1 can take as long as needed to be fixed, if any other Feature has a bug discovered we just revert it, and development can continue. Also, all the versions on all servers have the same code (which means, v1.0.3 in staging is the same as v1.0.3 in development). 

What is the problem then? Well, when Feat.1 is done, we have to revert the revert. And our history becomes a mess. Imagine what it would look like if Feat.2 and 3 also did reverts?

There is one other option that works with many costs. We cherry-pick the contents of the development branch into staging.

Now this one comes with a bunch of requirements to work. Let me point out a short list of things to be careful when going in this direction:

  1. When developing your feature branch, you must not merge the parent branch regularly (only when you are going to merge). This adds a risk that your feature branch and the parent branch will be very different when you merge, and your code will need a lot of adaptations to be merged.
  2. You must not squash the commits when merging to the parent branch. Otherwise, cherry-picking will be almost impossible.
  3. The version number in development no longer guarantees it has the same code in staging and prod. Notice that v1.0.2 in the development branch has Feat.2 and the faulty implementation of Feat.1, whereas v1.0.2 in staging only has Feat.2.
  4. Code in staging might become very different than code in develop over time. One thing to consider is to merge staging --> development on a regular basis.
But it also has the nice advantage that all the features can be deployed from development to staging independently, with little concern about bugs introduced by other features.

Alternatives for Testing Large Features

Finally, for very large features that we expect to find many bugs when testing, we can consider another approach: deploying a branch in a separate server and running tests there. You can configure this server to use all the dependencies ( database, storage, redis cache, etc) of the test server.


This works great and keeps our repo clean because you will only deploy to development once it has already been tested by the test team, with the extra cost of having to configure any other tool or client we use for our test with the new server endpoint. 

What are the cons? Besides the extra configuration for the test team, there is an extra cost for the new server. Also, if you do not have a pipeline ready to deploy it, there is the extra work of deploying it correctly. Basically: more work to do for this to work.

Conclusions

There is no "best" way of handling bugs found in tests. As a team, you have to figure out the approach that best suits the moment. If you have time, not very large features being developed, stuff that hardly has bugs or that bugs will be easily fixed, just wait for the fixes and deploy it. If the time to fix the issue is not clear, you have to choose between cherry-picking and reverting.

Notice that a great advantage here would be knowing in advance if the feature implementation is large/complicated/has high story points or not. That is one good use of those estimated story points in sprint planning. 😉


Comments

Post a Comment

Popular posts from this blog

Distributed Computing with Spark SQL (Coursera)

By
Image
Overview This course was a great positive surprise regarding the way the practical exams worked. You can think of this as a great introduction to Databricks, along with Spark, and a fact I think few people know: you can use Databricks for free if you create a community account. Spark is one of the most important tools (if not the most important) for large data processing (a.k.a. Big Data). The main advantage it provides over traditional data processing tools is the possibility of horizontal scaling, which makes scalability almost limitless. Spark is a basic tool for Data Engineers and one of the largest problems one can face when learning Data Engineering is the problems we are trying to solve are usually for big corporations with a huge pile of data, which often involves high costs to simulate such situations. This course is part of the SQL Basics for Data Science specialization. Here are the course details from Coursera: Pros As mentioned before, the best part of this course was the
Show me more »

Foreign Visitors in Brazil - 2005 to 2015 - Part II

By
Image
So this is the continuation of our previous analysis of foreign visitors from 2005 to 2015. The first thing we see is how the variable of interest (number of arrivals) behaves through the years. It is observable that there has been a slight increase in the number of foreign visitors since 2013. It is also possible to see how the accumulated visits are distributed through the continents on the graph below. From this graph we can understand that this increase is specially because of South America. For the other two continents with most visitors, Europe and North America, there has been a slight decrease in arrivals. Below we have the graph of arrivals per continent in this 10 year window. South America and Europe have by far the larger number of visitors. Next we observe how the sum of arrivals is ditributed through the months of the year. As expected, there is a larger number of arrivals during summer time in Brazil. However, there is an unexplained increase in July. So far, one possib
Show me more »