Software Design Problems: Wicked or Tame?

By John Dooley

There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.

—C. A. R. Hoare

One way to look at software problems is with a model that divides the problems into two different layers:

• “Wicked” problems fall in the upper layer: These are problems that typically come from domains outside of computer science (such as biology, business, meteorology, sociology, political science, and so on). These types of problems tend to be open ended, ill defined, and large in the sense that they require much work. For example, pretty much any kind of a web commerce application is a wicked problem. Horst W. J. Rittel and Melvin M. Webber, in their 1973 paper (1) on social policy, “Dilemmas in a General Theory of Planning” gave a definition for and a set of characteristics used to recognize a wicked problem.
• “Tame” problems fall in the lower layer: These problems tend to cut across other problem domains and tend to be better defined and small. Sorting and searching are great examples of tame problems. Small and well defined don’t mean “easy,” however. Tame problems can be very complicated and difficult to solve. It’s just that they’re clearly defined and you know when you have a solution. These are the kinds of problems that provide computer scientists with foundations in terms of data structures and algorithms for the wicked problems we solve from other problem domains.

According to Rittel and Webber, a wicked problem is one for which the requirements are completely known only after the problem is solved, or for which the requirements and solution evolve over time. It turns out this describes most of the “interesting” problems in software development. Recently, Jeff Conklin has revised Rittel and Webber’s description of a wicked problem and provided a more succinct list of the characteristics of wicked problems (2), in the article “Wicked Problems and Social Complexity”. To paraphrase:

1. A wicked problem is not understood until after the creation of a solution: Another way of saying this is that the problem is defined and solved at the same time. (3)
2. Wicked problems have no stopping rule: You can create incremental solutions to the problem, but there’s nothing that tells you that you’ve found the correct and final solution.
3. Solutions to wicked problems are not right or wrong: They are better or worse, or good-enough or not-good-enough.
4. Every wicked problem is essentially novel and unique: Because of the “wickedness” of the problem, even if you have a similar problem next week, you basically have to start over again because the requirements will be different enough and the solution will still be elusive.
5. Every solution to a wicked problem is a one-shot operation: See the preceding bullet point.
6. Wicked problems have no given alternative solutions: There is no small, finite set of solutions to choose from.

 Wicked problems crop up all over the place. For example, creating a word processing program is a wicked problem. You may think you know what a word processor needs to do—insert text, cut and paste, handle paragraphs, print, and so forth. But this list of features is only one person’s list. As soon as you “finish” your word processor and release it, you’ll be inundated with new feature requests: spell checking, footnotes, multiple columns, support for different fonts, colors, styles, and the list goes on. The word processing program is essentially never done—at least not until you release the last version and end-of-life the product.

Word processing is actually a pretty obvious wicked problem. Others might include problems where you don’t really know if you can solve the problem at the start. Expert systems require a user interface, an inference engine, a set of rules, and a database of domain information. For a particular domain, it’s not at all certain at the beginning that you can create the rules that the inference engine will use to reach conclusions and recommendations. So you have to iterate through different rule sets, send out the next version, and see how well it performs. Then you do it again, adding and modifying rules. You don’t really know whether the solution is correct until you’re done. Now that’s a wicked problem.

Conklin, Rittel, and Webber say that traditional cognitive studies indicate that when faced with a large, complicated (wicked) problem, most people will follow a linear problem-solving approach, working top-down from the problem to the solution. This is equivalent to the linear approach of the traditional waterfall model (4) shown in Figure 1:

Instead of this linear, waterfall approach, real wicked problem solvers tend to use an approach that swings from requirements analysis to solution modeling and back until the problem solution is good enough. Conklin calls this an opportunity-driven or opportunistic approach because the designers are looking for any opportunity to make progress toward the solution. (5) Instead of the traditional waterfall picture in Figure 1, the opportunity-driven approach looks like Figure 2.

In this figure, the jagged line indicates the designer’s work moving from the problem to a solution prototype and back again, slowly evolving both the requirements understanding and the solution iteration and converging on an implementation that’s good enough to release. As an example, let’s take a quick look at a web application.

Say a nonprofit organization keeps a list of activities for youth in your home county. The list is updated regularly and is distributed to libraries around the county. Currently, the list is kept on a spreadsheet and is distributed in hard copy in a three-ring binder. The nonprofit wants to put all its data online and make it accessible over the web. It also wants to be able to update the data via the same website. Simple, you say. It’s just a web application with an HTML front end, a database, and middleware code to update and query the database as the back end. Not a problem.

Ah, but this is really a wicked problem in disguise. First of all, the customer has no idea what they want the web page(s) to look like. So whatever you give them the first time will not be precisely what they want; the problem won’t be understood completely until you are done. Secondly, as you develop prototypes, they will want more features—so the problem has no stopping rule. And finally, as time goes on, the nonprofit will want new features. So there is no “right” answer; there is only a “good enough” answer. Very wicked.

Conklin also provides a list of characteristics of tame problems, ones for which you can easily and reliably find a solution. “A tame problem:

1. Has a well-defined and stable problem statement;
2. Has a definite stopping point, i.e., when the solution is reached;
3. Has a solution which can be objectively evaluated as right or wrong;
4. Belongs to a class of similar problems which are all solved in the same similar way;
5. Has solutions which can be easily tried and abandoned; and
6. Comes with a limited set of alternative solutions.”

A terrific example of a tame problem is sorting a list of data values:

• The problem is easily and clearly stated—sort this list into ascending order using this function to compare data elements.
• Sorting has a definite stopping point: the list is sorted.
• The result of a sort can be objectively evaluated (the list is either sorted correctly or it isn’t).
• Sorting belongs to a class of similar problems that are all solved in the same way. Sorting integers is similar to sorting strings is similar to sorting database records using a key and so on.
• Sorting has solutions that can easily be tried and abandoned.
• Finally, sorting has a limited set of alternative solutions; sorting by comparison has a set of known algorithms and a theoretical lower bound.

Realizing that most of the larger software problems we’ll encounter have a certain amount of “wickedness” built into them influences how we think about design issues, how we approach the design of a solution to a large, ill-formed problem, and gives us some insight into the design process. It also lets us abandon the waterfall model with a clear conscience and pushes us to look for unifying heuristics that we can apply to design problems.

<sup>Footnotes:
1. Rittel, H. W. J. and M. M. Webber. “Dilemmas in a General Theory of Planning.” Policy Sciences 4(2): 155-169 (1973).
2. Conklin, J. Dialogue Mapping: Building Shared Understanding of Wicked Problems. (New York, NY: John Wiley & Sons, 2005.)
3. DeGrace, P. and L. H. Stahl Wicked Problems, Righteous Solutions: A Catalogue of Modern Software Engineering Paradigms. (Englewood Cliffs, NJ: Yourdon Press, 1990.)
4. Conklin, J. Wicked Problems and Social Complexity. Retrieved from http://cognexus.org/wpf/wickedproblems.pdf on 8 September 2009. Paper last updated October 2008.
5. Conklin, J. (2008)</sup>

About the Author

John F. Dooley is the William and Marilyn Ingersoll Professor of Computer Science at Knox College in Galesburg, Illinois. Before returning to teaching in 2001, Professor Dooley spent nearly 18 years in the software industry as a developer, designer, and manager working for companies such as Bell Telephone Laboratories, McDonnell Douglas, IBM, and Motorola, along with an obligatory stint as head of development at a software startup. He has over two dozen professional journal and conference publications and four books to his credit, along with numerous presentations. He has been a reviewer for the Association for Computing Machinery Special Interest Group on Computer Science Education (SIGCSE) Technical Symposium for the last 36 years and reviews papers for the IEEE Transactions on Education, the ACM Innovation and Technology in Computer Science Education (ITiCSE) Conference, and other professional conferences. He has developed short courses in software development and developed three separate Software Engineering courses at the advanced undergraduate level.

Want more? This article is excerpted from the book Software Development, Design and Coding.  Get your copy today and learn the principles of good software design, and how to turn those principles into great code, with this introduction to software engineering.