The Random Sustainable Coder

Looking for good books? Here are two on software development and one on… randomness.

The Clean Coder: A Code of Conduct for Professional Programmers” by Robert Martin addresses what it takes to be successful in the long term as a software professional. First and foremost, a software professional takes responsibility for himself, his peers and his product. For example, he speaks the truth. He voices his opinion if a deadline will not be met. Martin has a series of conversations between different roles (such as programmer, project leader) which are pretty entertaining. There are lots of topics in this book that are important if you want to be long-lived and respected as a software professional. If you don’t have time to read the whole book, at least read the first chapter, which is gold.

Sustainable Software Development: An Agile Perspective” by Kevin Tate addresses what it takes to be successful in the long term when developing a software product. It is not unusual to see software products released after a year and then maintained for ten more years. This sets some specific requirements on the software, the development environment and the mindset of the developers. For example, the software must be able to accommodate change at reasonable cost and without breaking. In order to be able to deliver fixes and new features, the product must always be in good shape. Preferably good enough to be delivered every day, if so desired. This means the development process and the mindset of the developers must be heavily focused on quality. We must shift from defect detection (finding bugs) to defect prevention (avoiding bugs). Root cause analysis is key. Read the book, the message is great.

Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets” by Nassim Nicholas Taleb is a very interesting book! It is about randomness and how it affects our life. For example, how much of the success in your own career is due to chance? According to the author, if you’re a stock broker, it’s probably close to 100% (Taleb does not go easy on his fellow finance colleagues! :). If you’re a software developer, my gut feeling is that luck is much less significant. Re-live your life a million times, and your average life salary would probably still be around your current. Not so for the stock broker.

Taleb also describes an interesting scam. Send one million emails saying “the stock market will rise next week”. Send another million emails saying “the stock market will fall next week”. Next week, if the stock market rises, keep only the first million email addresses. To half of them, send an email saying “stocks will rise next week”, and to the second half “stocks will fall”. Every week, keep only the half where you predicted correctly. After 10 weeks you will have hundreds of people thinking you have super powers… This is called “survivor’s bias”. Taleb applies the exact same reasoning when a stock broker asks him for a job. Taleb won’t ask “How did you manage to do so well in the stock market in the past?”, but rather “How many like you were there from the beginning?”! This is a fantastic book, read it.

Characteristics of a Software Professional

At work, I have been challenged with the question “What are the most important characteristics of a software developer?“. This is a tough question, and no matter what you decide to include, you have to leave something out.

I’ve been part of software development projects in various companies. The successful projects teach you invaluable lessons. The dysfunctional projects even more so. Inevitably, a list of characteristics would include qualities I value and desire in my fellow colleagues, as well as characteristics I’d expect them to want in me. (So this is also a long todo list for me. ;)

To get some kind of structure, I decided on three main categories: Professionalism, Long-term code and Quality mindset.

Professionalism

Take responsibility

You are a professional developer, and professionals act responsibly. If things go wrong,
take responsibility. Understand why things went bad. Learn, adapt and make sure it
never happens again. When faced with a difficult choice, “do the right thing”. Optimize for the long run even if it results in more work today. Be a team player, even if this means saying no. Speak the truth.

Know your product

We’re part of a business. Without successful products, there will be no business.
Know your users and their needs. Use your product, use competitor’s products,
visit customers and watch them use your product (from purchase/download to
installation to day-to-day usage to upgrade to uninstall and so forth).

Continuous learning

Be humble and practice relentless inquiry. Question everything (since everything has potential for improvement), embrace questions on your code, realize others’ suggestions might improve your work, be happy other developers change “your” code
(it’s good enough to understand!). Improve yourself and others. Become a better developer by reading books, papers, blogs etc. Watch instruction videos, listen to podcasts, try new technologies, participate in open source development, discussion groups etc. Discuss your code and what you read with your peers. Talk to developers of other specialities. Teach others what you know well.

Long-term code

Communication

You write a line of code once, but it is read hundreds of times. Invest time to write code easy to read for others. Write code at the right level of abstraction, abstract enough for expressiveness but without hiding necessary detail. Adhere to design principles as
they capture proven ways to high-quality code. Apply design patterns to better communicate your intent.

Maintainability

Decouple the different parts of your software, on every level – sub-system, module, class and function. Write extendable code, so that you can add functionality with minimal change to existing code. Avoid technical debt, and repay debt as soon as possible. Interest has to be payed on all debt.

Proven functionality

Never deliver code unless you’ve proven it works. If you don’t test it, it will be faulty.
Write testable code. Make it testable from the start, later it will be too expensive. Automate your tests to run before check-in, after check-in and nightly. If the tests are not executed automatically, they will not be updated and soon be obsolete. Without automated tests, no-one will dare change any code. Write fast and reliable unit tests. Write realistic integration tests. For each new feature, write acceptance tests. Automate everything.

Quality mindset

Quality is your responsibility

You, as a developer, is responsible for the quality of the product. Other roles can help you spot problems or give you more time, but they cannot affect quality. Never ship anything without being certain of its correctness.

Find bugs early

Find bugs early in the development process. If a bug can be found by the developer, it should be. If you need tools, get them. If you need hardware, get it. If a bug is found late, understand why it was not found earlier. Fix the process so that bugs of this kind never slips through. Automate.

Fix it now

If you find a bug, fix it now instead of filing a bug report. Ask your colleagues to help out. You will save time. File bug reports on things that couldn’t be solved in half a day.
Do things properly the first time. If you don’t have time to do it right today, when are you ever going to find time? Give time estimates that allow you to produce quality products. Think about what is stopping you from being more productive. Fix it, and then move on to the next thing stopping you.

If you’re interested in these things, you should have a look at Poppendiecks’ “Lean software development” books, Senge’s “The fifth discipline“, Martin’s “Clean coder“, McConnell’s “Code complete“, McConnell’s video “World class software companies” and others. I also provide some resources in previous blog posts (e.g. videos and books).

What characteristics do you value in a software developer?

Sustainable Software Development, Part 1: Managing Technical Debt

Software projects sometimes go bad. The pace of development is not sustainable. To achieve sustainable software development, we need to keep our focus on what’s important: the long-term health and maintainability of our source code.

Robert Martin has written a nice article on Scrum projects, and how applying Scrum often start out with hyper-productivity. However, most projects slow down when code size increases. After a while, we might even experience that progress approaches zero as time approaches infinity (if the project is not discontinued before it :).

Why is that? Martin lists a number of activities that are well-known to the experienced developer, such as unit testing, continuous integration and code coverage measurements. Most developers would even agree that applying them will help retaining productivity over the long run. Still, the law of least resistance often takes us down a different path. A strong focus on feature growth is very seductive, as it makes everybody happy in the short term: the customer is happy about their new features, the product owner is happy when he can satisfy the market and the team is happy when everybody commends them for their work. At least for a short while. The lack of focus on quality (such as a lack of automated unit tests and a poor continuous integration setup) will catch up with you.

Technical debt are the things you know you must do but haven’t done yet: fixing a bug, adding more unit tests, refactoring, cleaning up etc. Just as normal debt, it has to be payed back. No surprise there. Unfortunately, just like a normal debt, technical debt also come with an interest rate. You pay interest every time you say “aargh, this would have taken me ten minutes had I refactored this module” or “aargh, I just spent half a day tracking down a problem caused by a known bug”. Discussing, planning or thinking about what must be fixed is paying interest. Soon, your house belongs to the bank. Software development like this is not sustainable – it will come to a halt.

To avoid this, and to achieve sustainable software development, we need to keep our focus on what’s important: the long-term health and maintainability of our source code. Just like the debt analogy, there’s a savings analogy. Invest some money in a savings account (or the stock market), and you will receive interest instead of paying. As a start, it is necessary to make an investment in good practices: fix things now, and use tools and techniques to ensure quality. You will be able to deliver new features for a long time and your velocity will stay high, or even increase over time.

Software Development is a Strange Profession

Software development is similar to other creative professions in many ways. For example, working with buildings as an architect involves planning, recognition and application of well-known patterns, problem solving and so forth. But software development is different from most other professions in one very important way.

Imagine you are a painter, an architect or a musician looking for a job. What kind of information would you supply in your job application? Obviously, you would show proof of your skills (photographs of paintings or buildings or recordings of music). As a software developer, what do you show? An application?

Assume it’s a very nice application (or a very bad one). Sure, you can draw some conclusions from the externally observable quality and behavior of an application. But development is normally a team effort, your code might be perfect while other’s isn’t and vice versa… Showing an application also assumes it can be shown. Your stuff might be a deeply integrated in a backend server somewhere, or worse.

A friend of mine brought up the example of an electrician. After a job well done, the wiring is normally not visible, somewhat like the programmer’s. Nevertheless, the electrician can still take photos before walls cover up their work. Unfortunately, the code you write is normally a well-kept secret of the company you work for (and might be off-limits in terms of photography :). The problem is, as a software developer you have nothing to show! This is not only a problem for you, but also for people trying to hire you.

As I’ve said before, your CV does not say much, since experience is not really an indicator of quality code. Admittedly, there are ways to prove your skills: doing a test task, participating in open source software development, creating a portfolio of sample code to show etc. But this doesn’t change the fact that software development is a strange profession. Can you think of a profession that has the same problem?

My Favorite Interview Question

Do you interview to hire a software developer? Maybe you’re being interviewed. In any case, this is my favorite question to determine how skilled you are.

What is the ultimate interview question? What question reveals whether or not a person is the one you’re looking for? Clearly, it depends on what you want. But even if you have a job advertisement outlining very specific technical skills, I argue in my previous post that, what you first and foremost need is someone that has a good sense on software structure. In object-orientation, structuring a program is really about understanding and applying object-oriented design principles. Now that we know what skills we’re looking for, it is easier to formulate a question.

I like evidence based interviews where we discuss what the potential hire did in the past. Thus, my favorite question is “Can you give me an example of good object-oriented code from your working career?”question answer Sometimes I even add “it doesn’t have to be code that you wrote”. You would certainly expect a professional programmer to be able to recall at least one good example from his working career. The point is not really to test whether or not he can remember code he wrote or read. Rather, it is to make sure he has an opinion. But most importantly, to have something to discuss. Anyone can do namedropping, so we need a concrete piece of code or a drawing to talk about. The upside with something that I didn’t choose is that the problem and solution domain (its terminology, background etc.) is known to my potential hire. That makes the interview situation less stressful. And I get to see if he is good at communicating ideas.

Surprisingly, this seems to be a tough question and I sometimes have to settle without an answer. Maybe it’s the pressure of the interview situation (hey, maybe it’s me!) but it reflects badly on the interview subject. If he can’t recall or don’t have an opinion on what is good code, how could I expect him to write any? So I have to move on. “How about bad examples of object-oriented programming?” This is a little bit easier. Everybody has opinions on other people’s code, and we can pick up from there: “Why is it bad? How can it be improved? What is the difference between bad and good?”. With luck, we get some examples to discuss.

I recently looked for decent examples as a basis for discussions around object-oriented programming. I came across this blog post by Reginald Braithwaite discussing the game of Monopoly. The main ideas are that Monopoly is well-known, but still complex enough to require the interview subject to discuss requirements. It also have some non-local properties, e.g. you can only build a house if you own all streets in that neighborhood. I like it. If the above fails, it will open up the discussion on object-oriented design principles in a nice way.

I think I will take a summer break from the writing now. A few weeks at least. See you!

 

Skills That Will Get You Hired

What is it that companies look for when they hire programmers? Surprisingly, they probably want something else than what’s in the job ad.

Job advertisements often focus on technologies (network programmingfor hire, windows programming, XML, you name it) or expert skills in a specific language. As a consequence, most CVs focus on the same thing. This is only natural, since people want to get hired. With otherwise equally strong candidates, the technology skills listed in the CV might affect who’s hired. But highlighting technology skills in a job ad is probably misguided, since I think these skills are not what companies really need. Let me explain what I mean.

In order to get a software system to do what we want, we need the above-mentioned technology knowledge. So, it’s definitively required, don’t get me wrong. But for the project to be successful over time, technologies are worth nothing unless the structure of the code is good enough for maintenance. (Somebody said that your code is in maintenance mode after the first line of code has been checked in. There is some truth in that.)

Also, acquiring a new piece of technology is not rocket science. Give it a couple of months, and you will have good enough knowledge. You will not become an expert, but there is no need for everyone to be an expert. To me, the skill to properly structure software is an essential one. Nevertheless, it is a rare one. Could it be that it is a skill hard to acquire? My point is this: avoid hiring people for knowledge that can be taught, without making sure he/she has what is essential. More specifically, what is that?

Here it is: I value people that write code that are testable, modular, readable and extensible. Without testability, I dare not touch your code. So I would have to throw it all away and start over, risk breaking it while making it testable or just dive in with my eyes closed. Nether is acceptable. With good modularity, the worst case scenario is that modules with unreadable code can be replaced or wrapped with tests. Modularity will also help readability and understanding. If you can understand the interfaces, understanding the internals become less important. Once we get down to internals of a module, readability becomes crucial. Only if I can read it (= understand it), can I maintain and extend it. Extensibility allows your software to meet customer’s needs and survive over time.

In object-orientation, there are design principles that will help you write testable, modular, readable and extensible software (see e.g. these three posts). If someone showed up with a good sense of how to write well-structured software using these design principles, he would certainly be a good hire. And conversely, if someone wasn’t convincing enough on his skills in writing structured software, he would be hard to hire. Regardless of his skills in other areas.

Outliers: The Story of Success(ful Development Careers)

Do you want to be an expert in your field? Do you want to be a superb programmer? If so, the good news is that there’s a simple plan that takes you there.

outliersI recently read Malcolm Gladwell’s book “Outliers: The Story of Success“. To me, the main message from the book is that hard work can make anyone into an expert in his field, no matter if it’s violin playing, ice hockey or… programming. The hard work is the bad news. According to the book, you cannot find a single “expert” (whatever that means), that has not spent at least 10.000 hours practicing. For example, not one of the top violinists have gotten to where they are by talent only. The fact that you don’t have to be born with a certain talent is kind of comforting. With 20 hours of practice per week and 52 weeks per year, anyone can be an expert in only ten years. :)

Unfortunately, working as a software developer for ten years will not automatically give you expert skills. Going to work and doing a good job is not sufficient. According to Gladwell, there needs to be a conscious and focused effort to improve. As a programmer, what can we do to improve?

First of all, I suggest you take every opportunity for practice during your workday. Make sure every day counts. You have the perfect setting: you work with code many hours per week, you have skilled colleagues to interact with, you have people from different fields for cross-pollination (management, sales, graphics design etc or different fields of engineering) and so forth. Make sure that you constantly think about your code and how to improve the way you write it. Seek out those that are better than you in whatever aspect. Make other people review your work. Discuss your design with others.

Work will only take you so far. Apart from practice, you also need theory. My favorite source of inspiration and knowledge is books. Most books I read challenge the way I think about code. Each author has his view on what constitutes good software. Be critical and incorporate the things you like into your code. Set aside an hour or a few minutes every day to read books, online articles and blogs (maybe even this one). Do not constrain yourself to just your field; problems and solution in other fields will also help.

Speaking of other fields: you should read the “Outliers” book, or you would not know why

  • Being born in January, February or March maximizes your chances to become a successful ice hockey player.
  • Asian children are better at math than western children.
  • Poor children’s reading abilities degrade over the summer while wealthy children’s reading ability improves.
  • There’s a school in Bronx where young children study from 7 am to 5 pm (and have 3 hours of homework every day) with very little summer vacation, and still, lots of children want to go there.

In short, I like the book.

Great Books for a Software Developer

I have received questions about good books, so I thought I would share some titles with you. First, my absolute favorites, should be mandatory reading for everybody (so go ahead and read them right away):

  • Growing Object-Oriented Software Guided by TestsGrowing Object-Oriented Programs Guided by Tests” by Steve Freeman and Nat Pryce (Amazon). My bible. :) A fantastic book about test-driven development. I touched upon it in my post Test-Driven Development Done Right.
  • Implementing Lean Software Development: From Concept to Cash” by Mary and Tom Poppendieck (Amazon). An amazing book on continuous improvement and customer focus in your software development organization. See also my post Lean Software Development.
  • Code Complete, 2nd edition” by Steve McConnell (Amazon). A great and practical book on how to write good code.

Really good books (read them when you finish the above :):

  • The Fifth Discipline: The Art & Practice of the Learning Organization” by Peter Senge (Amazon). A fantastic book on how to make people work together so that your organization becomes greater than the sum of the individuals. Not really a software development book, that’s the only reason why it didn’t make it to the “mandatory reading” list. :)
  • Design Patterns: Elements of Reusable Object Oriented Software” by Erich Gamma et. al. (Amazon). The Book on design patterns. A must-have.
  • Peopleware: Productive Projects and Teams” by Tom DeMarco and Timothy Lister (Amazon). A book on what is required to create an productive software development organization. Pretty interesting facts in there.
  • How to Win Friends and Influence People” by Dale Carnegie (Amazon). A classic about how to build good relationships and how to handle people.
  • One Minute Manager” by Kenneth Blanchard and Spencer Johnson (Amazon). A classic about how to manage people in order to get them as happy and productive as possible.
/www.amazon.com/Minute-Manager-Ph-D-Kenneth-Blanchard/dp/0425098478

Zero Tolerance: Writing Good Software Is Tough

I want to work with clean and readable code, with decent test coverage. Thus, when I run across some small flaw in the code (e.g. code duplication in some test scripts), it should be fixed. But I admit, sometimes I just convince myself “hm, I’m so close to getting this feature up and running, let’s fix this code after that”. This almost always give me more agony later.

So when I had the opportunity to participate in a new project with no dependencies on legacy code, I thought I would try something new, just for fun. Let’s call it zero tolerance: “when finding flawed code, fix it now. Flawed code could be anything: smelly code , confusing script names, non-descriptive function names, bad file structure, incomprehensible test cases, you name it. Let nothing pass!

Trying it out, my first observation was that it was painful! You have this new, fully functional feature within reach and this crappy code that requires a one-day refactoring appears. It is tough on your mental health to just ignore the easy-to-reach reward and do the really boring stuff. Especially when your threshold to what needs to be fixed is so low (actually, it’s zero). Sometimes it will feel like all you do is fiddle with old code.

Stick or carrot

Stick or carrot?

The upside is that all the punishment will make you learn (stick-style rather than carrot). You know that if you don’t concentrate and do things properly the first time, you will suffer later. And like I said, the suffering will inevitably happen when you are just about to write those last lines to get your system ready for a test spin. I would say it motivates you to produce code of higher quality from the beginning.

When I first started doing the zero tolerance thing, I expected it to get easier over time. The funny thing is, the amount of code that needs to be fixed seems to be constant over time! How is that possible? Maybe I am just sloppy and introduce lots of bad code? (Probably, but the punishment makes me improve! :) Maybe it is just impossible to foresee all the problems that may appear? Maybe. But I also think something else is at play.

Large software projects often grind to a halt. The complexity of the software becomes overwhelming and new features are delivered more and more slowly. Why is that? With increasing amounts of code, there will be increased opportunities for sloppy code. It will pile up. Without proper care, the amount of code that needs to be fixed will grow over time (probably much faster than the code size grows, the broken windows theory doesn’t help either). So maybe being constant over time is really a great outcome.

Like I said, I was hoping that the work load from code fixing would actually decrease over time. Maybe software is just hard (in terms of essential complexity), and there’s not much we can do about it. We will have to do our best not to make things worse by introducing too much accidental complexity. I will keep doing the zero tolerance thing for a while. I’ll report back on my findings if something interesting happens.

If you decide to try the zero tolerance approach, be warned: you should probably do it on a new piece of code. If you choose a piece of code that was written with a normal level of acceptance for design flaws and technical debt, you will probably lose your mind before you get any new functionality in there!

“We Train You to Be the Best Programmer You Can Be”

What could a company do to differentiate itself in order to attract talents? Many companies take pride in their innovative work environment. Benefits and salaries might be another way. But we are yet to see a company with an official policy saying “Come work for us. We will train you to become the best programmer you can be.“? You know of such a company? I would be happy to be proven wrong.

I’m not talking about having the employees take courses or read books all day long. I would rather see a collaborative effort to improve (yourself and the organization), such as a dedicated mentor for everyone, expert brain picking sessions, study circles (discuss a book, a bug, a piece of code or a paper together), knowledge transfer (weekly presentations of interesting topics), pair programming with an expert, software architects to discuss your design with, cross-functional training, panel discussions, required reading, you name it. All in the context of your current work assignment. Your personal development would be part of the company targets.

Many companies train their staff, but I would like to see it as an official policy. Such a company would certainly be able to attract talented people. The collaborative effort to improve would also benefit the company (e.g. improved co-operation, better code, fewer bugs, increased knowledge transfer). For the individual, working there would be a sign of quality.

I had the pleasure to have a coffee break discussion about this with James Coplien the other day. He said he both loved and hated the idea. It might be a bad idea since it does not bring any business value. But that aside, we both agreed it would be a fantastic place to work in.

 

Update (May 7, 2011): In the book “Implementing Lean Software Development” (see post), the authors talk about river companies: “The purpose of a river company, […] is to keep on flowing, that is, to stay in business and provide jobs over the long term.” and “The individual will deliver care and commitment in exchange for the fact that the company will try to develop each individual’s potential to the maximum.” This actually sounds a lot like what I described above. The term seems to have been defined in this article.