Testing it from the start

Example Project: https://github.com/alscube/TestingFromTheStart

This is an example of Test Driven Development at compile time.

As you read this blog and look at the sample code you'll notice that I am believer in Test Driven Development.   Years ago I started a company that created testing software to do regression testing.  I did it because I do not enjoy testing and re-testing software, I think it is the death of any programmer. 

Now with all the test frameworks available every programmer should be involved with testing their software.  There should be no excuse.  There are handful of frameworks and I'm sure everyone is going to choose their preferred package.  I've used Google Test, CPP Unit, JUnit, and QTest.   They essentially work the same so moving from one to another shouldn't be too difficult.

Test driven development is great at keeping you focused on what you are delivering and it lets you know immediately if something is broken.  The unit tests created should be compact and run quickly.   If possible they should run almost every time you do a local build and they should definitely run each time you do a full build.

The example code linked above was created with Qt.  It uses QTest as the test framework.  The project builds a static library, a dynamic library and a main app.   The library code is essential tested at compile time, as soon as each library is built it is immediately tested.  If the test passes the next library is built, if the test fails all compiling stops. 

The build sequence is as follows; 

    Static library is built 

    Static library Test App is built and executed.  

    If the static library test passes ....

    Dynamic library is built 

    Dynamic library Test is built and executed

    If that dynamic library test passes ...

    Main Project is built and executed

The project layout is:

SUBDIRS =

    ./AddLibrary/AddLibrary.pro

        SUBDIRS = 

            ./AddLibrary/AddLibrary.pro

                TARGET = AddLibrary

            ./AddLibraryTest/AddLibraryTest.pro

                TARGET = AddLibraryTest

    ./SubLibrary/SubLibrary.pro

        SUBDIRS = 
            ./SubLibrary/Sublibrary.pro
                TARGET = SubLibrary
            ./SubLibraryTest/SubLibraryTest.pro
                TARGET = SubLibraryTest

    ./MainProject/MainProject.pro

        TARGET = MainProject

I chose the Qt environment because it makes the testing easy.  In the project file a command is added to execute the just built file.  So the static library test project looks like:

    TARGET = AddLibraryTest

    CONFIG += c++14

    CONFIG += console

    CONFIG += testcase

    QT -= gui

    QT += testlib

    INCLUDEPATH = ../AddLibrary

    HEADERS += \

        AddLibraryTest.h

    SOURCES += \

        AddLibraryTest.cpp \

        AddLibraryTestMain.cpp

    macx {

        DESTDIR = .

        CONFIG -= app_bundle

        LIBS += ../AddLibrary/libAddLibrary.a

    }

    QMAKE_POST_LINK = $$DESTDIR/$$TARGET

Its this last line that makes it all work:

    QMAKE_POST_LINK = $$DESTDIR/$$TARGET

The QMAKE_POST_LINK tells the build environment to execute what it is assigned to.  In my case the TARGET app is my unit test.  It immediately runs at the end of a the test code build.

If you watch the compiler output window you'll see the code being compiled, then you'll see the test output:

........ rms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -mmacosx-version-min=10.14 -Wl,-rpath,@executable_path/../Frameworks -Wl,-rpath,/Users/user/Qt/6.2.3/6.2.3/macos/lib -o ./AddLibraryTest AddLibraryTest.o AddLibraryTestMain.o moc_AddLibraryTest.o   -F/Users/user/Qt/6.2.3/6.2.3/macos/lib ../AddLibrary/libAddLibrary.a -framework QtTest -framework Security -framework AppKit -framework ApplicationServices -framework Foundation -framework QtCore -framework DiskArbitration -framework IOKit   

./AddLibraryTest

********* Start testing of AddLibraryTest *********

Config: Using QtTest library 6.2.3, Qt 6.2.3 (x86_64-little_endian-lp64 shared (dynamic) release build; by Clang 13.0.0 (clang-1300.0.29.3) (Apple)), macos 12.6

PASS   : AddLibraryTest::initTestCase()

PASS   : AddLibraryTest::AddTenTest()

PASS   : AddLibraryTest::AddOneHundredTest()

PASS   : AddLibraryTest::cleanupTestCase()

Totals: 4 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms

********* Finished testing of AddLibraryTest *********

(and more code being compiled)

......... table_path/../Frameworks -Wl,-rpath,/Users/user/Qt/6.2.3/6.2.3/macos/lib -o SubLibraryTest SubLibraryTest.o main.o moc_SubLibraryTest.o   -F/Users/user/Qt/6.2.3/6.2.3/macos/lib /Users/user/Workspace/Projects/AlsCube/TestingFromTheStart/build/SubLibrary/SubLibrary/libSubLibrary.dylib -framework QtTest -framework Security -framework AppKit -framework ApplicationServices -framework Foundation -framework QtCore -framework DiskArbitration -framework IOKit   

/Users/user/Workspace/Projects/AlsCube/TestingFromTheStart/build/SubLibrary/SubLibraryTest/SubLibraryTest

********* Start testing of SubLibraryTest *********

Config: Using QtTest library 6.2.3, Qt 6.2.3 (x86_64-little_endian-lp64 shared (dynamic) release build; by Clang 13.0.0 (clang-1300.0.29.3) (Apple)), macos 12.6

PASS   : SubLibraryTest::initTestCase()

PASS   : SubLibraryTest::SubTenTest()

PASS   : SubLibraryTest::SubOneHundredTest()

PASS   : SubLibraryTest::cleanupTestCase()

Totals: 4 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms

********* Finished testing of SubLibraryTest *********

(then the main project)

........ utable_path/../Frameworks -Wl,-rpath,/Users/user/Qt/6.2.3/6.2.3/macos/lib -o MainProject main.o   -F/Users/user/Qt/6.2.3/6.2.3/macos/lib /Users/user/Workspace/Projects/AlsCube/TestingFromTheStart/build/MainProject/../AddLibrary/AddLibrary/libAddLibrary.a /Users/user/Workspace/Projects/AlsCube/TestingFromTheStart/build/MainProject/../SubLibrary/SubLibrary/libSubLibrary.dylib -framework QtCore -framework DiskArbitration -framework IOKit   

/Users/user/Workspace/Projects/AlsCube/TestingFromTheStart/build/MainProject/MainProject

MainProject add 50

MainProject sub 800

Another example is the FileStore SDK project.  You can download it from Github.  The sample project provided is the actual unit tests used to validate the FileStore I/O and record operations.   The FileStore library is built as a dynamic library, the test code is built and it validates the library.  There are four separate test files with a total of 93 tests.

If you have never done test driven development you'll need to get use to not writing a lot of production code up front.   The purists will tell you to write the test first then the code the test operates against.  The test code validates what you want to do and the production code is how you want to do it.   I still write the production code first, but only up to the point that it can do something useful, then I write the test.  Only when I have proven my general theory do I go back to the production code and start implementing the details and exception handling, and as I do each addition I go back and write the supporting test to prove the additional code.    If you end up doing this correctly you can achieve a very high level of code coverage.

When it comes to writing unit tests my approach is to write multiple unit tests per function to be tested, not a single unit test that tests all features and cases.  So you'll find that I have one or more positive tests, these test the function as it should work.  Then I'll have a series of negative tests, forcing it to fail.  If you find that your unit tests are becoming long, that may be a sign that you should break up the production code into smaller functions.

At first it will seam like you are writing a lot of extra code, but your code will function as expected and when a change comes a long and breaks things you immediately know it.