Unit Test Presentation at Edgeware

Everyone that wants to get serious with unit testing should follow the lead of Edgeware and dedicate half a day or so of their developers’ and testers’ time to identify strengths and weaknesses within the development organization. I was invited to present best practices of unit testing. We also touched upon continuous integration and future directions in terms of development excellence. It was fun and interesting and we had good discussions. :) Perhaps the slides (pdf) can give you some inspiration to improve your unit testing and development process.

Characteristics of a Software Professional

At work, I have been challenged with the question “What are the most important characteristics of a software developer?“. This is a tough question, and no matter what you decide to include, you have to leave something out.

I’ve been part of software development projects in various companies. The successful projects teach you invaluable lessons. The dysfunctional projects even more so. Inevitably, a list of characteristics would include qualities I value and desire in my fellow colleagues, as well as characteristics I’d expect them to want in me. (So this is also a long todo list for me. ;)

To get some kind of structure, I decided on three main categories: Professionalism, Long-term code and Quality mindset.

Professionalism

Take responsibility

You are a professional developer, and professionals act responsibly. If things go wrong,
take responsibility. Understand why things went bad. Learn, adapt and make sure it
never happens again. When faced with a difficult choice, “do the right thing”. Optimize for the long run even if it results in more work today. Be a team player, even if this means saying no. Speak the truth.

Know your product

We’re part of a business. Without successful products, there will be no business.
Know your users and their needs. Use your product, use competitor’s products,
visit customers and watch them use your product (from purchase/download to
installation to day-to-day usage to upgrade to uninstall and so forth).

Continuous learning

Be humble and practice relentless inquiry. Question everything (since everything has potential for improvement), embrace questions on your code, realize others’ suggestions might improve your work, be happy other developers change “your” code
(it’s good enough to understand!). Improve yourself and others. Become a better developer by reading books, papers, blogs etc. Watch instruction videos, listen to podcasts, try new technologies, participate in open source development,¬†discussion groups etc. Discuss your code and what you read with your peers. Talk to developers of other specialities. Teach others what you know well.

Long-term code

Communication

You write a line of code once, but it is read hundreds of times. Invest time to write code easy to read for others. Write code at the right level of abstraction, abstract enough for expressiveness but without hiding necessary detail. Adhere to design principles as
they capture proven ways to high-quality code. Apply design patterns to better communicate your intent.

Maintainability

Decouple the different parts of your software, on every level – sub-system, module, class and function. Write extendable code, so that you can add functionality with minimal change to existing code. Avoid technical debt, and repay debt as soon as possible. Interest has to be payed on all debt.

Proven functionality

Never deliver code unless you’ve proven it works. If you don’t test it, it will be faulty.
Write testable code. Make it testable from the start, later it will be too expensive. Automate your tests to run before check-in, after check-in and nightly. If the tests are not executed automatically, they will not be updated and soon be obsolete. Without automated tests, no-one will dare change any code. Write fast and reliable unit tests. Write realistic integration tests. For each new feature, write acceptance tests. Automate everything.

Quality mindset

Quality is your responsibility

You, as a developer, is responsible for the quality of the product. Other roles can help you spot problems or give you more time, but they cannot affect quality. Never ship anything without being certain of its correctness.

Find bugs early

Find bugs early in the development process. If a bug can be found by the developer, it should be. If you need tools, get them. If you need hardware, get it. If a bug is found late, understand why it was not found earlier. Fix the process so that bugs of this kind never slips through. Automate.

Fix it now

If you find a bug, fix it now instead of filing a bug report. Ask your colleagues to help out. You will save time. File bug reports on things that couldn’t be solved in half a day.
Do things properly the first time. If you don’t have time to do it right today, when are you ever going to find time? Give time estimates that allow you to produce quality products. Think about what is stopping you from being more productive. Fix it, and then move on to the next thing stopping you.

If you’re interested in these things, you should have a look at Poppendiecks’ “Lean software development” books, Senge’s “The fifth discipline“, Martin’s “Clean coder“, McConnell’s “Code complete“, McConnell’s video “World class software companies” and others. I also provide some resources in previous blog posts (e.g. videos and books).

What characteristics do you value in a software developer?

Black Box Programming

Why do software developers focus so much on the inside of the system when what we really want is to correctly implement the system as seen from the outside? Is it possible to first write the code for the external behavior, and then tweak the inside? Maybe, but we might need to rethink.

I’ve developed reactive system with asynchronous input for many years, such as telephony systems. Over time, I have become increasingly puzzled by the way we develop these systems. A few years ago, I realized what was bothering me.

In the development of a reasonably complex piece of software, there are at least three roles involved: requirements engineering, software engineering and quality engineering. The requirements people look at the system from the outside. They view the system as a black box and define the behavior of Black Boxesthe system by its incoming and outgoing signals (as well as non-functional requirements). Testers also look at the system from the outside. They inject signals and verify expected outgoing signals (as well as non-functional requirements). The software developer wants to implement a system that corresponds to the requirements. Thus, it would be natural for the developer to think of the system as a black box (ignore the internals!). He would start out by implementing the observable behavior, perhaps by describing the incoming and outgoing signals of the system as a state machine. For most non-trivial systems, this is not what we do.

Instead, the programmer focus on what is on the inside of the system. Implicitly, and perhaps scattered over hundreds of thousands of lines of code, various sub-routines define the logic and outgoing signals of our system. The externally observable behavior is a side-effect of these sub-routines. In any non-trivial system, it is almost impossible to verify the correctness by visual inspection. We make our best effort to test our software to ensure the correct behavior, with the cost that it incurs. Focusing on the inside of the system is only natural, since many factors affect the source code of our system (e.g. non-functional requirement such as performance, scalability, robustness etc.). Natural or not, it results in systems difficult to implement correctly. Thus, the question here is: can we do anything about it? Would it be possible to explicitly describe the externally observable behavior in a single place in our code while still being able to satisfy non-functional requirements?

What would this code look like? It would describe the logic of the system: incoming and outgoing signals and the necessary control flow along with some house-keeping data. Let’s call it the Black Box description of the system. It would not contain any implementation details (threading, database storage, performance tweaks, etc.) since implementation details are not necessary to describe the external functional behavior of the system. Everything would be described in the domain language. Let’s take an example: Assume we are implementing an ATM machine implementingATM this behavior. It would have incoming and outgoing signals towards the user interface (user input and text on the display), the bank’s server (requests and responses) and the ATM machine itself (card inserted, eject card etc.). The number of failed PIN attempts would be stored in a variable (house-keeping data). The “Too many invalid PINs” transition (here) would read this variable.

We write the Black Box code so that it is executable in isolation. This means the Black Box code must only talk to abstractions, and never directly to code that contain implementation details of the system (networking, platform specifics and optimizations etc.). Executable in isolation also means it will be unit testable in isolation. By testing the behavior of the Black Box code, we can verify the logic of the system. Obviously, other kinds of tests are required to verify the full implementation (state stored in databases, networking behavior etc., not to mention non-functional requirements). An executable Black Box would be a tremendous advantage. We would not need an implementation to be able to try out our system. Testers could start verifying and integration tests could begin very early on. We could also do rapid prototyping.

Many systems can be implemented simply by a state machine like the one in the ATM example. But large systems tend to be much more complex than that. First of all, we need to be able to address non-functional requirements. There are also some implementation issues to consider. I wouldn’t be able to list all challenges, but to get a feeling for how some of the issues can be addressed, let’s mention a couple of them:

  • Asynchronicity: For example, we are writing a system that requires authentications. As an implementation detail, we may choose to query another machine over the network. Thus, asynchronous results are inevitable. We don’t want the Black Box to reveal this (since our authentication procedure does not concern the end user). In our implementation, we could adapt our Black Box state machine by introducing a sub-state machine where we wait for the response from the other machine. This require us to be able to extend or substitute a state with a sub-state machine.
  • Performance: What if multiple threads are involved in the execution of the Black Box? We might have to divide the state machine so that parts of it is executed in one thread and parts of it in another thread (or even in different processes or machines). Here, too, we need asynchronous communication between threads, much like the above. We also need the ability to execute only parts of the state machine.
  • State: Assume the flow of the state machine does not depend only on input signals but also on some other piece of data. For example, it could be a database query that answers whether a user is registered or not. Somewhere in the Black Box description we might call a function isUserRegistered(). In the implementation, we will use a real database. When executing the Black Box during testing, we let isUserRegistered() return pre-determined values for different test cases, very much like a mock object.

Flower SpiralTo implement larger systems, we would combine the Black Boxes of our sub-systems into a larger whole. We would build a hierarchy of Black Boxes. Black Boxes would communicate with each other through incoming and outgoing signals, which translates into asynchronous signals or synchronous function calls, whichever is most appropriate. The combined system would also be unit testable. Unit testing the combined system would exercise the sub-systems, since the combined system’s behavior relies on the behavior of the sub-systems.

The idea of a Black Box description is definitively not new. There are tools for model-driven design that come very close to what is described above. For example, in Mentor Graphic’s BridgePoint, you can describe your Black Box behavior as a state machine and generate customized code by using what is called a model compiler. I’ve seen several successful projects built on BridgePoint, so the concept seems viable. But most programmers feel most comfortable when the code is at the center of things. You want full control of your code. This may also be a contributing reason why model-driven design has not taken off. So, the question here is really what can be done without advanced tools.

I developed a framework for Black Box Programming a few years ago. It addressed the challenges described above (e.g. replacing a state or transition with a sub-state machine, execution of parts of a state machine, unit testing) and had a couple of nice features (e.g. random walk in the state machine, execution of the unit tests towards the real system). So I think it is possible to develop software like this. The question is just if it is practical. I think the best way to get a feeling for Black Box Programming is to try it out. If you have a project in mind that can be open-sourced, get in touch and we’ll try it out together!