small constant too).
So of course it's much better as it's O(N * T) vs O(N * T * M) if
you would test everything after each commit.
This means that test suites are good to prevent some bugs from being
committed and they are also quite good to tell you that you have some
bugs. But they are not so good to tell you where some bugs have been
introduced. To tell you that efficiently, git bisect is needed.
The other nice thing with test suites, is that when you have one, you
already know how to test for bad behavior. So you can use this
knowledge to create a new test case for "git bisect" when it appears
that there is a regression. So it will be easier to bisect the bug and
fix it. And then you can add the test case you just created to your
test suite.
So if you know how to create test cases and how to bisect, you will be
subject to a virtuous circle:
more tests => easier to create tests => easier to bisect => more tests
So test suites and "git bisect" are complementary tools that are very
powerful and efficient when used together.
Bisecting build failures
~~~~~~~~~~~~~~~~~~~~~~~~
You can very easily automatically bisect broken builds using something
like:
-------------
$ git bisect start BAD GOOD
$ git bisect run make
-------------
Passing sh -c "some commands" to "git bisect run"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For example:
-------------
$ git bisect run sh -c "make || exit 125; ./my_app | grep 'good output'"
-------------
On the other hand if you do this often, then it can be worth having
scripts to avoid too much typing.
Finding performance regressions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is an example script that comes slightly modified from a real
world script used by Junio Hamano <<4>>.
This script can be passed to "git bisect run" to find the commit that
introduced a performance regression:
-------------
#!/bin/sh
# Build errors are not what I am interested in.
make my_app || exit 255
# We are checking if it stops in a reasonable amount of time, so
# let it run in the background...
./my_app >log 2>&1 &
# ... and grab its process ID.
pid=$!
# ... and then wait for sufficiently long.
sleep $NORMAL_TIME
# ... and then see if the process is still there.
if kill -0 $pid
then
# It is still running -- that is bad.
kill $pid; sleep 1; kill $pid;
exit 1
else
# It has already finished (the $pid process was no more),
# and we are happy.
exit 0
fi
-------------
Following general best practices
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is obviously a good idea not to have commits with changes that
knowingly break things, even if some other commits later fix the
breakage.
It is also a good idea when using any VCS to have only one small
logical change in each commit.
The smaller the changes in your commit, the most effective "git
bisect" will be. And you will probably need "git bisect" less in the
first place, as small changes are easier to review even if they are
only reviewed by the committer.
Another good idea is to have good commit messages. They can be very
helpful to understand why some changes were made.
These general best practices are very helpful if you bisect often.
Avoiding bug prone merges
~~~~~~~~~~~~~~~~~~~~~~~~~
First merges by themselves can introduce some regressions even when
the merge needs no source code conflict resolution. This is because a
semantic change can happen in one branch while the other branch is not
aware of it.
For example one branch can change the semantic of a function while the
other branch add more calls to the same function.
This is made much worse if many files have to be fixed to resolve
conflicts. That's why such merges are called "evil merges". They can
make regressions very difficult to track down. It can even be
misleading to know the first bad commit if it happens to be such a
merge, because people might think that the bug comes from bad conflict
resolution when it comes from a semantic change in one branch.
Anyway "git rebase" can be used to