Software Project Management

This is a course about software development "in the large" and "for real." That means, software development

• of very large systems (thousands of lines of code, also known as KLOCs)
• by multi-programmer teams
• to meet client needs
• with deadlines and budgets

The Problem

The field of software engineering arose a number of years ago in response to a perceived software crisis: While there seems to be a doubling each year in the number of software applications demanded, the size of such applications, the complexity of such applications, and mission criticality of such applications, there has been almost no increase in programmer productivity or program quality.

To put it bluntly, much of what is developed is:

• late
• buggy
• and not very useful

The crisis arises because standard programming techniques, such as those you've learned at Northwestern, are totally inadequate for real world software needs. The consequences have been much worse than getting a C-:

• Slipped delivery dates in large software projects, from operating systems to air traffic control system to spreadsheets, have cost companies millions of dollars, driving some out of business.
• Bugs in software controlling medical devices and traffic systems have killed people!

The Cause

These problems arise because fundamental, critical questions were either not asked or not properly answered:

• How long will it take to build this system? How many people will have to work on it? How do you know?
• Exactly what will it do? How usable will it be? How do you know?
• How do you know it will work correctly? How can you prove it?
• How hard will it be to change? How do you know?

Everything you know is not enough

All of you have probably tried to answer the first set of questions in some form for various course assignments, and perhaps in some programming job. Chances are that for at least some of your projects, you weren't even close. Scale the error in your estimates for a one-person three-week project to a five-person six-month project and you'll see how bad it can get.

Most you have probably never had to answer the remaining questions. What your course projects were supposed to do was decided by someone else (the professor or the textbook) and no one really used the results anyway.

Step back and look at the entire software development process, which is typically broken down into these phases:

• determining what the customer needs (requirements analysis)
• determining what should be built and how it should be organized (system design)
• implementation of the system design
• testing, including validation, verification, etc.
• delivery, including integration into existing systems, training, monitoring, etc.
• maintenance, including bug fixes, adjusting to platform changes, extensions and upgrades

Chances are good that your programming courses have only dealt with implementation and perhaps a little design. Implementation, the part where you've spent countless hours, and the part that probably defines what software development is for you, is in fact only a small part of the overall process.

In fact, in most cases, you can tell a well-run project from one that is doomed by how much of the effort is spent on implementation. If implementation accounts for more than 20% of the project effort, the project is in trouble. Most likely, there's

• no testing plan, so little testing will occur and the product will be bug-ridden
• no integration plan, so the product may not even install and run at all
• no design documentation, so changes to fix bugs will cause more bugs and probably undo some functionality
• no requirements analysis, so what's delivered won't actually support what the customer needs to do

Approaches to Solutions

There are many, many different approaches to resolving these problems, some complementary, some contradictory. These approaches can be organized under two key themes.

Better Management

Management approaches including the following techniques:

• better metrics for estimating how much effort it will take to develop software
• better methods for tracking the development efforts, especially across large teams
• methodologies for software development, that
  • specify a recipe for the development process and
  • provide tools to improve thinking and record keeping, such as
    • formalized notations
    • paper forms
    • CASE (computer aided software engineering) tools

Quality Assurance

Quality approaches include the following techniques:

• rigorous requirements analysis, to determine what the client needs and how the business works
• design notations, to specify data and functional relationships in a formal but language independent way
• program correctness tools
• sophisticated test suite development techniques
• rapid prototyping and usability testing very early in the development process

Course Goals

In no way can a one-quarter undergraduate course cover everything in what is a very broad field with a wide variety of subareas, a wide variety of often contradictory methodologies, and a lot of debate about works and what doesn't.

Therefore the goals of this course are modest but concrete. I want to make you better software engineers because you will

• know what the big problems are and understand their seriousness
• recognize these problems in your own software development situations
• be able to resolve these problems in a well-reasoned manner using sound methods
• be able to learn about and apply more structured methodologies as required

Course Activities

To that end, this course will consist of a number of activities and challenges:

• Testing and code review: You'll produce an in-depth test suite for code modules you've written for other courses or jobs. You also participate in structured walkthroughs on such code.
• Requirements analysis: You'll create a requirements analysis document for a medium size project, using standard UML notations.
• Project plan: You'll create a risk-driven iterated project development plan for the project.

Comments? Send mail to Chris Riesbeck.