Characteristics of a Software Professional

At work, I have been challenged with the question “What are the most important characteristics of a software developer?“. This is a tough question, and no matter what you decide to include, you have to leave something out.

I’ve been part of software development projects in various companies. The successful projects teach you invaluable lessons. The dysfunctional projects even more so. Inevitably, a list of characteristics would include qualities I value and desire in my fellow colleagues, as well as characteristics I’d expect them to want in me. (So this is also a long todo list for me. ;)

To get some kind of structure, I decided on three main categories: Professionalism, Long-term code and Quality mindset.

Professionalism

Take responsibility

You are a professional developer, and professionals act responsibly. If things go wrong,
take responsibility. Understand why things went bad. Learn, adapt and make sure it
never happens again. When faced with a difficult choice, “do the right thing”. Optimize for the long run even if it results in more work today. Be a team player, even if this means saying no. Speak the truth.

Know your product

We’re part of a business. Without successful products, there will be no business.
Know your users and their needs. Use your product, use competitor’s products,
visit customers and watch them use your product (from purchase/download to
installation to day-to-day usage to upgrade to uninstall and so forth).

Continuous learning

Be humble and practice relentless inquiry. Question everything (since everything has potential for improvement), embrace questions on your code, realize others’ suggestions might improve your work, be happy other developers change “your” code
(it’s good enough to understand!). Improve yourself and others. Become a better developer by reading books, papers, blogs etc. Watch instruction videos, listen to podcasts, try new technologies, participate in open source development,¬†discussion groups etc. Discuss your code and what you read with your peers. Talk to developers of other specialities. Teach others what you know well.

Long-term code

Communication

You write a line of code once, but it is read hundreds of times. Invest time to write code easy to read for others. Write code at the right level of abstraction, abstract enough for expressiveness but without hiding necessary detail. Adhere to design principles as
they capture proven ways to high-quality code. Apply design patterns to better communicate your intent.

Maintainability

Decouple the different parts of your software, on every level – sub-system, module, class and function. Write extendable code, so that you can add functionality with minimal change to existing code. Avoid technical debt, and repay debt as soon as possible. Interest has to be payed on all debt.

Proven functionality

Never deliver code unless you’ve proven it works. If you don’t test it, it will be faulty.
Write testable code. Make it testable from the start, later it will be too expensive. Automate your tests to run before check-in, after check-in and nightly. If the tests are not executed automatically, they will not be updated and soon be obsolete. Without automated tests, no-one will dare change any code. Write fast and reliable unit tests. Write realistic integration tests. For each new feature, write acceptance tests. Automate everything.

Quality mindset

Quality is your responsibility

You, as a developer, is responsible for the quality of the product. Other roles can help you spot problems or give you more time, but they cannot affect quality. Never ship anything without being certain of its correctness.

Find bugs early

Find bugs early in the development process. If a bug can be found by the developer, it should be. If you need tools, get them. If you need hardware, get it. If a bug is found late, understand why it was not found earlier. Fix the process so that bugs of this kind never slips through. Automate.

Fix it now

If you find a bug, fix it now instead of filing a bug report. Ask your colleagues to help out. You will save time. File bug reports on things that couldn’t be solved in half a day.
Do things properly the first time. If you don’t have time to do it right today, when are you ever going to find time? Give time estimates that allow you to produce quality products. Think about what is stopping you from being more productive. Fix it, and then move on to the next thing stopping you.

If you’re interested in these things, you should have a look at Poppendiecks’ “Lean software development” books, Senge’s “The fifth discipline“, Martin’s “Clean coder“, McConnell’s “Code complete“, McConnell’s video “World class software companies” and others. I also provide some resources in previous blog posts (e.g. videos and books).

What characteristics do you value in a software developer?

Replay: Recreate Every Single Bug

Do you ever spend lots of time trying to understand and recreate a bug scenario? Is there a bullet-proof way to reproduce every single bug? By logging the right information, it should be possible.

Reproducing bugs in complex systems is often hard. Even if the use case that caused the bug is explained in detail, this may not help much. Other factors, such as database state, configuration, timers, network activity and randomness, affect code execution. Without exact knowledge of these factors, you cannot determine which path was taken through your code. Often, debugging information is written to a trace log file to mitigate the problem. However, this is intrusive and clutters the code. Furthermore, by the time a bug is found in a live system, it is too late to add more tracing to the code. We need something else.

As said above, in order to reproduce a bug, we need to know exactly which path was taken through our code. In order to do that, we need to know exactly what decision is taken for every possible branch (if, for, switch etc.). In certain circumstances, this might actually be possible.

Recording the Input

First, let’s assume we have a module without concurrent data access (either single-threaded, by explicit synchronization or by design). Second, vhs cassettewe identify incoming events to the system, such as calls on a public API, GUI user interaction or network packages. They are the entry points to your code. Third, we identify all other external sources of data that might affect the execution. For example, calling a readFromDatabase function will return some data. The use of this data will affect the code execution. Thus, we treat all reads from the database as input to our module. The same goes for configuration, random values and all the other factors mentioned earlier. Combined, we think of the incoming events and the data from external functions as the input to our module.

Fourth, after identifying all module input, we introduce a mechanism to eavesdrop on the incoming data. For each event (e.g. callback userClickedButton or call to a public API function), we store the function arguments to a file. Let’s call this file the interaction log. Similarly, for each external function call (such as a database read), we store the return value or exception thrown.

Replaying From File

In order to reproduce the scenario of the bug, we replay the events from the file by injecting them as function calls into the module. The module will execute its code until it reaches an external function call such as readFromDatabase. Instead of calling the function, we retrieve the return value (or exception) from the file and return (or throw) that instead.

Now, there are a couple of challenges when implementing this approach. First, we’ve restricted ourselves to code that never accesses data concurrently. There’s probably nothing we can do about this. Anyway, from my experience, it is better to organize the software to avoid these concurrency problems since the alternative is just too painful (see post on concurrency).

Second, we want the calls from our module to an external function (e.g. readFromDatabase) to either call a real function (e.g. in the database implementation) or to replay from file. Obviously, we don’t want our module to be aware whether we are replaying or not. In object-orientation, we can achieve this through sub-classing an interface. The module talks to the interface, and behind the interface, there’s either a real entity (the database) or something that replays from file. Thus, all calls from your code to external functions must go through an interface, and never to a concrete implementation directly.

Third, external function calls (such as a database read) have arguments. What if the external function decides to manipulate one of the arguments? For example, it could call a setter method or change a public member. We would have a real problem. The replayed execution would not call the setter method, and the system state would not be equivalent to the bug scenario. Now, to me, manipulating the arguments of a function is poor style. Doing it in an API (for e.g. a database system) is even worse. So hopefully, situations like these are rare. But when they do arise, we will have to restructure our code slightly if we want to be able to replay.

Implementing the Replay Functionality

Java has a very handy mechanism to support the implementation of the replay functionality: reflection. We might have a large number of interfaces through which we call external functions. Nevertheless, using reflection, we can create a single wrapper class that can handle all interfaces. Lets denote it LoggingWrapper. We would create an instance of LoggingWrapper and supply it with an instance of the real class (e.g. the database implementation). We would give the LoggingWrapper object to our module, and our module would think it is talking to the real entity (the database). When our module calls an external function (e.g. readFromDatabase), the LoggingWrapper would forward the function call to the real entity (database) and then log the return value (or exception thrown) to file. If we don’t want to log to file, we would not create a LoggingWrapper. Thus, we would not suffer any performance penalty.

When we want to replay from file, we create a ReplayWrapper and give that to our module. From some other class (e.g. EventReplayer), we would read an event and its arguments (e.g. “the user clicked button X”) from the file and call the corresponding function on the module. When an external function is called (e.g. readFromDatabase), the ReplayWrapper would read a return value (or exception) from file and return it (or throw the exception). As an extension, we could also verify the integrity of the interaction log while replaying. The downside is that this require some extra information to be recorded in the log (such as the full name and argument values of each function call). An integrity check would be able to detect a number of things, such as if the arguments to an external function call differ from when the log was written.

You could imagine implementing the replay functionality per module. But a replay implementation seems complex enough to be non-trivial. It would be useful with a general purpose Replay framework. For fun, I have started sketching on one. Time will tell if/when it will be in good enough shape to be released to the public. All design/code/idea contributions are welcome. :)

Zero Tolerance: Writing Good Software Is Tough

I want to work with clean and readable code, with decent test coverage. Thus, when I run across some small flaw in the code (e.g. code duplication in some test scripts), it should be fixed. But I admit, sometimes I just convince myself “hm, I’m so close to getting this feature up and running, let’s fix this code after that”. This almost always give me more agony later.

So when I had the opportunity to participate in a new project with no dependencies on legacy code, I thought I would try something new, just for fun. Let’s call it zero tolerance: “when finding flawed code, fix it now. Flawed code could be anything: smelly code , confusing script names, non-descriptive function names, bad file structure, incomprehensible test cases, you name it. Let nothing pass!

Trying it out, my first observation was that it was painful! You have this new, fully functional feature within reach and this crappy code that requires a one-day refactoring appears. It is tough on your mental health to just ignore the easy-to-reach reward and do the really boring stuff. Especially when your threshold to what needs to be fixed is so low (actually, it’s zero). Sometimes it will feel like all you do is fiddle with old code.

Stick or carrot

Stick or carrot?

The upside is that all the punishment will make you learn (stick-style rather than carrot). You know that if you don’t concentrate and do things properly the first time, you will suffer later. And like I said, the suffering will inevitably happen when you are just about to write those last lines to get your system ready for a test spin. I would say it motivates you to produce code of higher quality from the beginning.

When I first started doing the zero tolerance thing, I expected it to get easier over time. The funny thing is, the amount of code that needs to be fixed seems to be constant over time! How is that possible? Maybe I am just sloppy and introduce lots of bad code? (Probably, but the punishment makes me improve! :) Maybe it is just impossible to foresee all the problems that may appear? Maybe. But I also think something else is at play.

Large software projects often grind to a halt. The complexity of the software becomes overwhelming and new features are delivered more and more slowly. Why is that? With increasing amounts of code, there will be increased opportunities for sloppy code. It will pile up. Without proper care, the amount of code that needs to be fixed will grow over time (probably much faster than the code size grows, the broken windows theory doesn’t help either). So maybe being constant over time is really a great outcome.

Like I said, I was hoping that the work load from code fixing would actually decrease over time. Maybe software is just hard (in terms of essential complexity), and there’s not much we can do about it. We will have to do our best not to make things worse by introducing too much accidental complexity. I will keep doing the zero tolerance thing for a while. I’ll report back on my findings if something interesting happens.

If you decide to try the zero tolerance approach, be warned: you should probably do it on a new piece of code. If you choose a piece of code that was written with a normal level of acceptance for design flaws and technical debt, you will probably lose your mind before you get any new functionality in there!