- The Difference between Agile and Waterfall
- Agile 101
- Misconceptions about Agile
- An Overview of Scrum
- Low-Cost Release Planning
- Teams in Charge
- Agile Architecture
- Benefits of Agile and Scrum
An easy way to understand the difference between Agile and waterfall is to compare Agile to a traditionally run project. In that, you initiate the project, plan the whole project, do most of the design, then do all the development, test it all, and finally release it and hope the customer likes it. After you’re done, you hold a “lessons learned” meeting to discuss better ways of doing things in the next project. This is called a “waterfall” project because each step is completed before the project moves down closer to the end goal:
In Agile you do all those steps, too. But after initiating the project much the same way, in most Agile approaches you do the planning, design, development, testing, release, and lessons-learned for a tiny piece of the final functionality in days or weeks. I think of each iteration as a “mini-waterfall,” as illustrated below:
By the way, Agile is not a “method,” as detailed in a moment. It is a philosophy and set of principles common to various methodologies that existed prior to the statement of the philosophy.
Almost all traditional or “waterfall” projects fail to deliver what was originally planned by the original planned date at the original cost. Every source I have consulted on this since becoming a project manager in the 1990s has supported this statement. The word “fail” is unfair, however. The idea that waterfall ever could do that is a myth. The simple truth is, no human can predict the future. You and I are not gods.
Consider this data quoted from a journal article about the ability of a group of experts to predict demand for electronics products:
|Future prediction||Forecast accuracy|
|1 month||± 5%|
|2 months||± 20%|
|3 months||± 50%|
|Beyond||Toss a coin|
This dates to 1994, when waterfall was the only kind of project management most people knew. In its Project Management Body of Knowledge (PMBOK), the Project Management Institute states firmly that changes to a project plan will result from execution of the project. Every project manager is taught how to “rebaseline” a plan, providing new date, scope, and/or budget expectations as new information arrives. That information ranges from revised customer requirements to better ways of accomplishing work, learned by doing that work. Other possibilities include changes in resources, changes in priorities, disruptions in supply, turnover of key individuals, and new innovations on the market. Recognizing this, I have long argued as a project manager and teamwork consultant:
Change by itself is neither good nor bad. Change is inevitable. The question is, do you manage it, or let it manage you?
Sadly, most companies have built their governance and supporting systems on the Waterfall Myth. Ponder if you will the “triple constraint” in traditional project management, meaning scope, schedule, and budget. Many visuals put quality in the middle, like this:
Sometimes referred to as the “Iron Triangle,” the term has lost favor because executives misunderstood the “Iron” in the title to mean the shape of the triangle remains as-is when the project manager does their job well. This is simply untrue. The point is to explain the impact of change. In any triangle, changing the length of one leg changes the shape of the triangle. The area changes as well, unless you lengthen one of the other legs.
When you increase scope, cost and/or time must go up. Cut cost, which on any project is mostly people, and scope must go down and/or time must go up. To decrease time, you must either decrease scope or increase cost to add people (directly or through outsourcing). The only other option in each case, as the figure above makes clear, is to decrease quality.
The further into the future you get, the less that future ends up looking like it did at the start. If a demigod looked down from the heavens at a new project, he or she could state how long it was actually going to take regardless of the project management method used. Waterfall almost always promises a date sooner than that, wastes a lot of labor hours on planning work that will never get done, and puts people through significant stress at the end due to the inevitable changes. True Agile approaches like FuSS promise no date up front, yet please the customer when the “demigod date” is reached. In fact, the final date in any properly run Agile project will likely beat the date waterfall would deliver, because fewer labor hours are on unnecessary planning, unused development, and unwieldy change control.
One driver of what we now call Agile methods was software people tired of getting pushed to meet dates that were impossible to predict. Months of planning in hopes of long-term predictability resulted instead in achingly slow releases of software that no longer met the customer’s needs, and still failed to meet the plans.
Full Stack Scrum™ does not predict a delivery date until that delivery is virtually certain. Because customers and the other business units that support them need some prior warning to implement a delivery, this system does predict with 80% confidence what scope will be delivered a quarter in advance. Some project managers and many executives don’t want to admit this, but that is as good or better than any waterfall project plan if quality isn’t sacrificed. The reality is, release dates are rarely firm more than a couple months in advance in waterfall. Per the PMBOK, a good practice for waterfall is to update the schedule every time a change is approved. Every time I saw this done in a disciplined manner in waterfall companies, this moved that release date regularly until the last 30 to 60 days. FuSS™ provides the same predictability with less planning/replanning time.
“The problem with predictive measurements is that they often do not reflect reality,” the Project Management Institute says in its 2017 Agile Practice Guide. “For example, project leaders describe the project as ‘90% done.’ At that point the team tries to integrate the pieces into a product. The team discusses missing requirements or surprises, or finds that the product does not integrate the way they thought it would.” In fact, PMI continues, the project often goes from “green” status to “red” one month before the release date “with seemingly no warnings…”
In practice, most waterfall companies remain in denial about delays of major releases until then, at which point some upper manager is forced by the facts to “bite the bullet” and tell the customers the bad news. Then everyone affected by the change in the customer and supplier companies has to re-plan everything in short order—sometimes more than once. Meanwhile, workers have been burning themselves out to meet a deadline that ends up moving anyway. In doing so, they introduce more mistakes, and managers often cut testing. Thus quality is lower, and customer satisfaction takes a hit because their schedule and scope expectations were not met. None of that happens in Full Stack Scrum. Bad news is inevitable, but FuSS never delivers a bad surprise, because it never hides anything from the customer.
Unfortunately, most middle and upper managers came up in waterfall environments awash in the Waterfall Myth, and that is all they know. Thus upper managers in non-Agile companies often insist on delivery dates for specific scope in a waterfall fashion from all teams, Agile or not, and expect the original numbers to be met. The concepts of Agile seem like chaos to these folks. Hence methods have been proposed to create long-term release planning in Agile projects, resulting in all the same problems as waterfall. FuSS—and from what I hear about failed attempts to scale Agile in general—will not work well in companies whose executives remain married to the Waterfall Myth and insist on “commitments” to deliver by specific dates beyond the next sprint.
I prefer the compassionate view that resistance to Agile, like resistance to the unknown in general, is due to lack of information. Most executives are intelligent people trying to balance the best interests of their customers, companies, and workers. I hope to convince you in the rest of this chapter that adopting an Agile mindset across your enterprise is the easiest way to accomplish that balance.
You may have heard terms like “Agile” and “Scrum,” but not be sure what they meant. That’s understandable; there is a lot of confusion on the Web, and many people treat the words as synonyms. Having given a lot of “Agile 101” talks to executives and sales people and engineers, I can reassure you most did not realize one term is a subset of the other—and the subset came first!
I like to say that Agile is a “philosophy,” or a set of principles, not a “method” of project management. Various alternatives to waterfall that were more responsive to change have been bubbling up in manufacturing, supply chain, and software development since at least the 1980s. In 2001, experts in some of these methods for software met in the ski town of Snowbird, Utah, USA. A common feature of their methods was delivering working software code quickly and improving it in small batches instead of trying to plan and deliver the whole product at once. They came up with four statements of priorities and 12 related principles common to their methods, and documented these as the “Manifesto for Agile Software Development,” commonly shortened to “Agile Manifesto.” There are competing manifestos for different fields at various stages of revision and acceptance, but the software Manifesto is by far the best-known:
Although the Manifesto and principles use the word “software” because they were written by software engineers, I believe you can replace that word with “product(s)” or whatever you create for customers. Agile has been used for deliverables in a wide range of disciplines, as I will discuss in a bit.
I’ve heard people say things like, “We’re not supposed to document,” or, “Agile means we don’t have to follow a process.” Not true, as the last sentence in the Manifesto shows. But Agile puts a greater emphasis on people talking to each other and dealing with change as it comes. As the American Management Association has put it, “In the agile environment, the PM emphasis is moved from planning to execution.”
The Manifesto’s authors added “Twelve Principles of Agile” that go into more detail about how their methods put the Manifesto’s priorities into practice. Again, if your team does not produce software, replace that word with your own deliverables as you read these:
- “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
Instead of waiting for months to release new functionality to end users, Agile methods do so in as little as days. Even if these deliverables are held until they can all be given to the customer in a bigger “release,” a fully tested “potentially releasable product” is produced at the end of each shorter period of time called an “iteration” or “sprint.”
The emphasis on customer satisfaction comes up over and over in discussions of Agile, and this is a critical point to bear in mind. Project success depends, above all else, on making the customer happy. Meeting the Iron Triangle doesn’t matter if that doesn’t happen. And satisfaction is a strong correlate to the customer’s willingness to buy from you again.
- “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”
In traditional project management, the customer often does not see a working product until months have passed. The desire to meet the Iron Triangle is paramount, so the team will spend months just planning, if necessary, in hopes of a short and accurately predicted execution phase. Meanwhile, the customer may have changed their minds about some requirements, creating significant rework and delaying project completion. The team may have realized there are better ways of meeting the requirements. And when the customer sees the deliverable, they may realize things important to them at the start no longer are, or identify new things they want.
Because of this, waterfall project teams impose formal change management processes. These take time and effort and may result in a change being rejected as too large or too late. Meanwhile the teams have wasted time doing work the customer decides is not valuable. In Agile, a customer can see a functioning deliverable quickly, and change their mind just as quickly. This cuts way down on wasted time for the team and ensures the customer ends up with the product or service they actually wanted… no matter what they initially asked for!
- “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”
I’ve already covered a lot of the benefits to shorter cycles. Also, estimation of work is much more accurate when limited to weeks instead of years. As explained later, some predictability beyond a single sprint is necessary for customer satisfaction in some settings. Plus, teams can get the personal satisfaction of producing something tangible far sooner than in traditional projects.
Another advantage the Manifesto writers may not have been aware of is the power of the deadline in speeding up work. Researchers have learned that teams with deadlines months in the future tend to wander aimlessly until the halfway point, and then get their acts together. Prior to my introduction to Agile, I recommended managers of teams that did not have deadlines create artificial ones via continuous improvement projects. Iterations of a month or less keep up the sense of urgency. However, FuSS includes techniques to ensure people do not burn out in the face of repeated deadlines.
- “Business people and developers must work together daily throughout the project.”
By relying on a contract; a relatively static requirements list; and formal change requests, waterfall disconnects the project team from the humans for whom they are providing value. Agile asks the customer, or at least a representative in regular contact with the customer(s), to actively participate in the project. That way the team gets feedback regularly to ensure it is meeting the customer’s current and future needs.
This approach goes a long way toward managing customer expectations. As we will see, those expectations are critical to satisfaction. Agile customers have the same view of progress as the teams, and understand their own impact upon it.
- “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”
Academic research into persuasion shows that an individual who comes to believe in an idea no longer requires outside motivation. They will work just as hard to accomplish that idea as they would if the idea had been theirs. Research also shows that the conditions people work in, from physical environment to policy constraints, have a greater impact on their behavior in a specific situation than personality traits. They can only perform at their best when they have the resources they need, are not blocked by policies or politics, and are only being told what is needed… not how to accomplish it.
- “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”
Conversations in person allow you to read an individual’s body language and facial expressions, cutting down on misunderstandings. They are more time-efficient than e-mail or chat sessions because responses are quicker and do not require multiple interruptions. Agile, and studies on teams in general, recommend team members have direct physical access to each other for that reason.
However, the reality of the modern business world is that many companies force teams to work virtually across multiple time zones. In that case, those teams should encourage regular use of media that allow as much “nonverbal” communication as possible, such as video chats and phone calls instead of e-mail or instant messaging. Members should also be close enough in terms of time zones to easily interact every day, as discussed further down.
- “Working software is the primary measure of progress.”
As stated in the first book on Scrum for software, “Measure work items done, not time spent per task.” Nothing else matters if you don’t provide something the customer can use, and providing that usable product is the quickest way to add value for the customer. Everything else you track, if it does not directly contribute to this goal, wastes time on administration that could be used to provide value to the customer.
- “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”
When people are forced to work long hours over long periods, they burn out. A vast amount of research shows those people make more mistakes, get slower in hour-by-hour output, get sick more often, and are more likely to quit. Agile focuses on realistic and ethical workloads in exchange for predictable short-term deliveries at high quality.
- “Continuous attention to technical excellence and good design enhances agility.”
Being responsive to change does not mean skipping design and quality. To the contrary, because these activities occur in each iteration and resulting deliverables are viewed by the customer earlier, the team has no choice but to keep quality “top of mind.” Regularly releasing products also makes fixing old defects more of a burden, encouraging members to find ways to reduce them up front.
- “Simplicity—the art of maximizing the amount of work not done—is essential.”
When the customer’s wants are a list of line items in a spreadsheet written months ago, it is too easy for project teams to veer off course into what they think the customer wants, or worse, what they think is the better way to go as they learn more about the project. This can create an awful surprise for everyone when the customer sees the final product and doesn’t want the “other stuff,” again adding rework and a lot of wasted time.
Related to the previous principle, when a Scrum team gets customer defect (or “bug”) reports, they have to take time out of their next iteration to go back and fix them. No one likes re-covering old ground, and the time required can put the team’s current-iteration promises at risk. Agile teams have very personal motivation to do only what the customer wants, in the easiest way to maintain.
- “The best architectures, requirements, and designs emerge from self-organizing teams.”
A consistent fact from decades of scientific research into group productivity is that “empowered” teams that make most decisions on their own outperform similar teams directed by a boss or dominated by a single expert. Based on my teamwork research, I believe this principle is the single most powerful reason for the success of properly implemented Agile methods. Agile not only allows but prescribes team empowerment.
As practiced, unfortunately, this is also the most corrupted principle. Companies often allow managers to overrule decisions made by teams. Worse, organizations that make money by selling certifications or placing people in jobs have misled companies into thinking the facilitation roles in the most popular Agile methods are full-time jobs. As a result, people in those roles, understandably trying to justify their pay, fill the vacuum by making decisions by themselves that are supposed to be team decisions. In short, they turn back into classic “team leaders.” This is a waste of payroll money and eliminates the psychological value of self-organization.
- “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”
By meeting after every iteration to review what went well and what didn’t, the team catches more mistakes and improves its operations earlier than standard project teams do. This saves significant time, hassle, and therefore money over the course of the project.
Among the many myths in business is the belief that Agile is something different from project management. Let’s go to the experts to explore that myth.
The Project Management Institute is the primary certification body for project managers. I am a long-time member, and proudly hold its best known certification, mistakenly associated with waterfall only, the Project Management Professional® (PMP). Although PMI is guilty of some of the same sins I have accused the Agile Industrial Complex of committing, the certifications are more stringent, and, critically, you have to have actually done the role—you can’t just take a short class and test and be considered “certified.”
The institute publishes its collective wisdom as the Project Management Body of Knowledge or PMBOK. I see in job ads a common misconception that the PMBOK is a “method.” Instead, Section 1.1 of the book states that it merely describes “good practices,” and continues:
“Good practice” means there is general agreement that the application of the knowledge, skills, tools, and techniques can enhance the chances of success over many projects. “Good practice” does not mean that the knowledge described should always be applied uniformly to all projects; the organization and/or project management team is responsible for determining what is appropriate for any given project.
Therefore, nothing in the PMBOK contradicts Agile. Yes, you choose to leave many parts out, but the same is true for any project. The remaining parts are all used in an Agile system—just in a different way than you may be used to. As an example, check out this graphic from the PMBOK:
The cost of a project needs to be justified regardless of the method used, and that is part of the “Initiating Processes.” However, the graphic clearly shows the “Planning” and “Executing Processes” recurring throughout the project in a cycle, with ongoing “Monitoring and Controlling Processes.” Some of the “Closing Processes” are done multiple times in an Agile project (with each release of a deliverable), but others are only done once, like closing out the project in the accounting system. Notice, too, that the title says this is only “One Approach” to one project.
As conclusive evidence that Agile is a form of project management, PMI has a certification in Agile, which I also hold: the PMI Agile Certified Practitioner (PMI-ACP). And the latest version of the PMBOK, Edition 6, is packaged with the Agile Practice Guide quoted above.
Agile is associated with software to the degree that many hardware designers don’t think it applies to them. To the contrary: The 1986 paper that first applied the term “Scrum” to business, “The New New Product Development Game,” was based on development of six physical products: copiers, cameras, a computer, and a car! The method called ‘Kanban” and now considered part of Agile started in Toyota manufacturing plants.
According to a Dutch researcher, “agility is the capability to rapidly reconfigure a manufacturing system for efficient production of new products as they are introduced. Agility is often equated with rapid response manufacturing and mass customization, as these concepts all aim to produce exactly what customers want… Agile manufacturing provides mechanisms to react quickly to changing markets, to produce high quality products, to reduce lead times and to provide a superior customer service, in a dexterous way.”
To feed that kind of manufacturing, supply chain experts have recognized the need for agility as well. “Companies that focus on agility are market-sensitive and will profit by exploiting their supply chains to rapidly and cost effectively respond to unpredictable changes,” wrote three professors at The Center for Engineering Logistics and Distribution in 2007.
Applying the FuSS principles to hardware design and development projects merely requires some changes of “sprintly” expectations, as detailed throughout this site. You cannot design an entire circuit in one sprint. You can determine the logical place on the board for a particular capacitor, given what you now know, and demonstrate a diagram or 3-D model to stakeholders for comment. As one researcher noted in reporting a study of Scrum hardware companies, “These product versions, or ‘protocepts,’ can be computer-generated 3-D drawings, virtual prototypes, crude models, working models, or early prototypes. The result of a done sprint, in this context, may not be a working product, but it is something physical that the customer can respond to.” The introduction of computer modeling and 3-D printing techniques mean hardware design processes can move much closer to software development cycles.
One veteran hardware engineer I worked with resisted the concept, but begrudgingly agreed to serve in an Agile role for a team of utility meter hardware engineers. This was in part due to his new boss, who successfully used Agile in cellular equipment design in a previous job. A year later the engineer admitted that earlier he did not want to do it. “But now I wouldn’t go back,” he said.
The PMI Agile Practice Guide mentioned above says, “Examples of people encountering high-uncertainty work include software systems engineers, product designers, doctors, teachers, lawyers, and many problem-solving engineers.” Agile methods have been employed by each of these. Nontechnical organizations are gaining the benefits, too. Marketing and Human Resources professionals have promulgated their own versions of the Agile Manifesto, and Scrum has been applied to public relations teams. Nonprofits can easily apply Scrum to longer-term deliverables from grant-writing to supplying new services, and Kanban to their daily service provision. I have seen Agile methods help high-level managers complete cross-functional strategic planning and process change efforts. There are even books about using Agile to run your family! I knew a Scrum Master who found it radically effective at getting the kids engaged in household activities.
Partially in response to the failed rollout of the HealthCare.gov site in 2013, the United States began hiring Silicon Valley experts to improve its IT services. They have been “spreading the agile development model to far corners of the US federal government.” Having worked with government teams from the local to federal levels, I can confirm both that government is slow to adopt better practices and that it can learn.
Alistair Cockburn, co-founder of the Crystal Clear method and a well-known author on Agile, used iterative methods to rebuild his house. He gave the Agile Advice blog an example: “Rather than excavate the whole basement, we decided to excavate only 1/3 of the basement. This gave us the basement entrance we needed, and left open the question whether we would extend the excavation or build a side wing in the next project… Three years later we still have no plans to extend the floor space, either above or below ground.”  Excavating the whole thing would not have saved money at the time, he said, and as it turns out, would have wasted a lot of money, since they decided they didn’t need the extra space. Cockburn, by the way, started his career as a hardware engineer.
The Agile Practice Guide provides a related example: “builders may want to show a finished room or floor of a building before they continue with the remainder of the building… The customer is able to see and approve the style, color, and other details, allowing adjustments to be made before further investments of time and money are made.”
I used Kanban to stain my back deck. I cleaned, treated, washed, and stained one small section at a time. As someone who does not enjoy home improvement, I found it remarkably stress-free. When other priorities or weather prevented my working on it, I didn’t have to worry about it or redo any steps.
Well, I had to once, because I failed to re-check the weather forecast, which I should have under Full Stack Scrum!
There are many different approaches to implementing Agile, such as Crystal, Extreme Programming (XP), and Rational Unified Process. Some are more focused on people’s interactions, while others are more technical. In practice, smart Agile coaches borrow liberally from the different approaches to blend a mix that best suits a specific project’s environment and needs. But it helps to start with one method as a framework, get that running smoothly, and then go hunting for new techniques to address specific situations.
Scrum is arguably the most popular approach. The word “scrum” refers to a big huddle of players on both teams in the sport of rugby. While drafting this site in a coffee shop, I chanced to meet members of the women’s rugby club from North Carolina State University. They explained that after a rules violation, eight players on each team gather and link up arm-to-arm. The teams then link with each other head-to-head, the ball is thrown underneath the scrum, and everyone uses their legs to try to push the ball out to a player on their team.
A member of the Wolfpack added something she said would make sense for this site. “The tighter and stronger the scrum is, the better it works,” she said. So true! Scrum is a method for getting teams to work tighter together and thus be stronger. By “team” I refer to a small group, at least three and preferably no more than eight people, with a mix of skill sets and roles but a common responsibility or purpose.
Very little in Scrum is required. Ken Schwaber and Mike Beedle laid out some basic concepts, principles, roles and meetings in their seminal book, Agile Software Development with Scrum. But Scrum has evolved in many ways. The method I present here is one way to do Scrum, not the one way. Some Scrum coaches will disagree strongly with parts of my method. But it is informed by my nearly two decades of working with empowered teams. Most importantly, it has proven reliable for moving teams quickly from waterfall or no project management to delivering consistently on their promises in as few as three months. This has been true for dozens of teams across multiple companies. Nonetheless, please bear in mind that statements I make about “Scrum” refer to my method, not necessarily Scrum in every instance.
This topic, “An Overview of Scrum,” takes you through the basics of generic Scrum while pointing out some specifics of FuSS. The goal is to introduce the system and provide information for evangelizing its adoption within your organization. It is not intended to give you all the information you will need to implement the system. Later chapters take you through everything here in greater detail.
In Scrum, an iteration or “sprint” is a short period of time in which the team tries to plan, design and deliver a set of working, fully tested deliverables. Each sprint is intended to result in a “potentially releasable (or shippable) product.” That is, whether or not you choose for business reasons to release the output to a customer, you could. The output is fully developed, tested, bug-fixed, and documented. For new or non-software projects, the output at least is tangible—not just ideas, for example, but documented decisions—and could be shown to a customer for comment. The sprint lasts one to four weeks, and sprints are repeated until the customer says the project is done. Once a team settles on a good sprint length for its circumstances, it should not change the cycle again without very good reasons. By sticking to one sprint length, the number and quality of deliverables the team can produce in a sprint becomes more predictable.
The basic unit of work in Scrum, adopted from Extreme Programming, is the “user story.” The story captures a requirement similar to an old-fashioned “use case.” A standard user story format is:
As a [type of user], I want to [do this] so that I can [achieve this purpose].
This format makes sure the team members are focused on the customer, understand what the customer wants, and know why it is important. Often knowing the purpose can influence the final design of a deliverable. The story also serves as a means for communicating with a customer on those points. In the ideal Agile situation, that person is an actual customer outside the company. More often he or she is someone inside the company who gets requirements from external customers, such as an account manager, product manager, or business analyst.
As you may have gathered, a critical difference between typical project requirements and user stories is that each story must be small enough to complete within a sprint. Indeed, a team will typically tackle multiple stories within a sprint in order to keep everyone busy. (We’ll discuss larger types of requirements called “epics” later.)
User stories may be recorded on note cards stored on a vertical surface like a bulletin board or whiteboard someplace where everyone on the team can easily see them. Some Agile teams sit together in the same room at least part time—often called a “commons,” “team room,” or “war room”—in which case a story board would be on a wall there. Software applications for managing Agile projects typically re-create a paper story board. They can display stories as icons that look like note cards; in text rows in tables; or the user’s choice of either. When I use the word “story” in this site, I will mean the three-part story statement. I’ll add the word “card” in reference to tracking the story within your tool, whether that is recorded on a physical note card or a digital representation.
Each story also needs “Acceptance Criteria.” This is a short statement, preferably measurable, by which the team can prove to the customer or their representative that the team achieved what it was supposed to. In FuSS it gets added below the story on the story card, which may also include high-level technical notes, risks, and dependencies.
Many teams assign “story points” as a size estimate, often using the first of a set of numbers called the “Fibonacci Sequence”: 1, 2, 3, 5, 8, and 13. Points do not measure anything. They express the Scrum team’s educated guess as to how big a story is compared to any other story. The size is not just a function of the mass of work, but also its complexity and/or risk, so this is not a time estimate in the waterfall sense. In a major departure from most Agile coaches, I no longer use points in my system. I mention them here because people who are familiar with Scrum will be expecting them. (For an explanation of my reversal on points, see “Why use task hours instead of story points?” on page 400.) Also, teams already using Scrum with stable story point velocities can implement the FuSS release planning methods without changing from that.
Put all that together, and you might end up with a story card like this:
Stories are stored initially in a section of the story board or application called the “Product Backlog.” This is the collection of all stories the team thinks as of now will need to be delivered at some point in the project, but not yet assigned to a time period. Those at the top are stored in the order they are likely to get done. However, the Product Backlog is regularly revised such that it may look completely different within weeks of starting the project. Stories are added, revised, deleted, and re-ordered regularly until scheduled into a sprint.
Scrum teams regularly meet to plan and review their work. This occurs in five types of meetings, often called “ceremonies,” that teams may schedule in different ways:
- Grooming Sessions—In these, the team develops a thorough understanding of each story and, in FuSS, a draft set of the tasks needed to complete it. This session can be a regularly scheduled meeting or scheduled as needed. Many teams conduct “grooming” during the Planning Ceremony instead, the more traditional approach.
- Planning Ceremony—On the first day of a sprint in Full Stack Scrum, the team members meet to:
- Determine what stories they will commit to completing in the sprint.
- Finalize and volunteer for tasks.
- Estimate how long each task will take in labor hours.
- Ensure they are not committing to more labor hours than each member has available during the sprint, and reduce the story commitment if needed.
- Daily Standup Meeting or “Scrum”—Every day between the Planning and Demonstration ceremonies, the team holds a very short meeting where each member answers only these three questions:
- What did you do since our last meeting?
- What do you plan on doing before the next meeting?
- Do you have any “blockers”—human or technical reasons you cannot complete a task?
- Demonstration Ceremony—On the last day of the sprint, the team shows what it accomplished to the customer and interested stakeholders to get input for future work.
- Sprint Retrospective—The team looks back on the sprint to answer these three questions:
- What went right in the sprint, in terms of results, process, or teamwork?
- What went wrong in the sprint?
- What should we do differently in the next sprint?
In Full Stack Scrum, the sprint starts when the Planning Ceremony ends, and usually ends when the “Demo” begins. Some teams take a day off between the two, meaning for example that a “four-week” sprint is really three weeks and four working days (19 labor days). Regardless, the cycle starts all over with the next Planning Ceremony. This figure summarizes the sprint meetings:
The customer—and technically, anybody in your company—is free to attend any meeting except the Retrospective, and is expected at the Demonstration Ceremony. During grooming, and of course the “Demo,” the customer is considered to be a source of vital information and thus a full “participant.” In all other cases, they and anyone else not on the team is only an “observer” and will be politely reminded not to speak unless asked a question. In trainings I jokingly refer to the old saying that children “should not speak unless spoken to.” Along with the limited questions in the Daily Standups and “Retros,” this is a way Scrum keeps its meetings as short as possible and builds team cohesion.
There are a number of standard reports and charts used in tracking Scrum team progress. The primary indicator is the “Burndown Chart.” This is a bar or line chart showing how many tasks or labor hours are yet incomplete in the sprint, and possibly how many stories have been accepted (see the figure below).
In FuSS, during each Planning Ceremony, the team agrees on a final list of tasks, volunteers for them, and applies labor-hour estimates to each. The Burndown Chart totals those estimates and calculates how many hours must be completed each day of the sprint to finish them all by the end, portrayed using an “ideal burndown line.” Each day, team members estimate how much time they think is left on the tasks they work on. The chart compares the total of these “to-do” hours to the ideal line.
To use an overly simple example: If you estimated 100 hours’ worth of work and used a two-week sprint (10 working days), the ideal burndown line would drop by 10 hours per day. Say after Day 3 the members had reduced the total remaining by 30 hours, regardless of how many labor hours that took them. The bar for that day would be touching the line, indicating good progress. If they only reduced it by 23 hours, it would be seven hours over the line. This would indicate the team is in danger of leaving some stories incomplete by the end of the sprint.
This nearly ideal Burndown Chart shows the actual results achieved in a sprint by a software team I worked with:
The darker bars on the left are remaining to-do hours, and the ones on the right are the total story points of completed stories (this company used points). The team hit the work hard on Day 1, kept steadily at it throughout the sprint, and only added a few tasks near the middle, meaning they had done a good job of thinking through the tasks during grooming and planning. The estimated hours for the added tasks explain the protruding bar on Day 8. Because their estimates were conservative, those additions did not prevent members from completing all of their original and added hours, and therefore they completed all of their stories.
I think one advantage to Scrum is that each day each team member gets to choose what they are going to work on. At the same time, the other team members get better visibility into those choices, and thus can provide input when those choices impact their own. Everyone also can easily see whether progress is being made on tasks they need done. If Joaquin is waiting on Terrance to complete a task so Joaquin can start on one of his, he knows to bring it up when nothing has been done a week into a sprint and Terrance says nothing about it in the scrum. (When I capitalize “Scrum” I am referring to the method; in lower case, “scrum” is just an alternate term for the Daily Standup meeting.)
The overhead for this is really low for the individual team member. At the start of the day, you might look at a list of your tasks to decide what sprint work to do, and you dive in on the task of choice when ready. Before or during your scrum, to make sure the Burndown Chart is up-to-date, you would go to the story board or application. For each task you worked on since the last ceremony, in FuSS you would update:
- The state of the task, such as “In Progress” or “Review.”
- How many hours you think are left (the “to-do” hours).
During the Daily Standup, other members may ask you to focus on tasks they are waiting on, but of course you have the same right. If anyone is falling behind in their sprint hours, you and your teammates have the right to ask why. Ultimately, though, what sprint work you do on a given day, and when in the day you do it, is up to you. People unaccustomed to others knowing what they do each day have objected that Scrum micro-manages them. I think this day-to-day freedom, and the fact members create and take accountability for tasks themselves instead of having those imposed by managers, disproves this objection.
Team Members with Extra Duties
Typically Scrum defines only two specific roles. The “Product Owner” (PO) serves as the voice of the customer in the customer’s absence and ensures the project delivers value for both the customer and the company. The PO has primary responsibility for gathering customer requirements, converting them into user stories, and prioritizing those stories. Although other schemes can be used, the primary method is the “rank order” of the stories in the Product Backlog. The team does its grooming and planning by working from the top of the backlog down. The PO is a full member of the team and participates in all of the meetings including the Retrospective. This role is often filled by a business analyst or technical expert.
The “Scrum Master” (SM) is the meeting facilitator and helps the team follow the processes it has agreed to. Also, if a blocker is identified in a Daily Standup, ensuring it is addressed quickly becomes the SM’s top priority. Either the PO, the SM, or both will handle any traditional project manager duties still required in Scrum that cannot be covered by other team members via user stories.
Note that each of these roles are part-time jobs. The PO will spend most of his or her time performing the person’s job title work (such as engineering work). Most of the SM’s duties takes place during the ceremonies, which the SM would have to attend anyway. Team members usually clear their own blockers. Thus the SM preferably is a member of the team with other primary duties who also handles the SM role. But he or she can be a full-time SM working with three or four teams at once. Hiring an SM for a single team is a waste of company money and guarantees the SM will do things to keep busy that he or she should not be doing for an empowered team (see page 46).
To illustrate, I downloaded a checklist of the things one consultant thinks a Scrum Master should do beyond the meeting facilitation and blocker-clearing all Scrum writers agree are part of the job. It is referenced by some Agile writers as a justification for one full-time SM per team. I went through it to see how many items I could identify as amenable by one of these short-term or part-time fixes:
- Fixable with one-time formal training by the SM or someone else (like a tool administrator), with minor coaching afterward.
- Caused by poorly enforced processes, mostly fixed during ceremonies through better discipline.
- Problems resulting from poor team organization prior to starting Scrum work, prevented or fixed by running one-off exercises in this site.
- A problem a good coach can usually solve with one conversation.
- A problem no one SM can solve by themselves because they are shared across teams, thus requiring only a part-time effort by each.
The answer was, “all of them.” One entire section is clearly “shared” and, in most larger organizations, would be driven by a cross-team facilitator or enterprise coach. Furthermore, most of the above actions should be done during Scrum ceremonies the SM would have to attend anyway, or through continuous improvement user stories during the early sprints. All of the checklist items are addressed during a FuSS implementation, most prior to starting sprints, and have been handled without issue by part-time or multi-team SMs I trained. Granted, much more of the SM’s time is dedicated to learning the system in the first few sprints, but that is true of everyone on the team—and hardly justifies a permanent full-time SM for the team.
This comes back to the critical point that neither the Product Owner nor the Scrum Master must be seen as the “boss” of the team. Per the Agile Manifesto, Scrum teams are self-organizing. The PO and SM are equal members of the team with everyone else. They only have additional authority in very specific and limited areas. The PO can override team decisions on priority and story completion on behalf of the customer, and the SM must take control of meetings and hold members accountable to the team’s practices. But otherwise everyone has an equal voice, and thus equal responsibility for the team’s successes and failures. The team is supposed to be a “one person, one vote” democracy for most decisions.
For multiple-team programs that deliver work in multi-sprint “releases,” FuSS has a few additional roles:
- Customer (with a capital “C”)—A customer-facing job title such as a product manager or solution architect who fulfills the customer duties in companies too large for actual customer(s) to participate in Agile planning. (A single team may need an official Customer, too, if more than one person provides it with requirements.)
- Agile Release Manager—Equivalent to the Scrum Master role, but for release planning meetings and cross-team blockers.
- Architect—A subject-matter expert who provides technical guidance to all teams, often filled by system architects and former functional managers.
- Agile Liaison—A representative of non-Agile or third-party organizations that provide support to, or need something from, the Agile teams.
A program only needs one Agile Release Manager, and if the programs are small, that person can handle more than one. (In this context, a “program” is both the work and the people doing it to support a major initiative, product, or service, usually over years and requiring multiple projects.) Most programs large enough to need release planning will have more than one person in each of the other roles. The first three roles must be assigned full-time to the program(s) they support, just like the teams. The Agile Liaison role is filled by someone in another organization, so by definition that person only participates part-time and may support multiple programs.
You may be thinking a role is missing: Agile Coach. Speaking as someone who often holds that job title, I do not believe in permanent Agile Coach positions. When Agile is running smoothly, only minor corrective coaching is needed, easily handled by the Scrum Masters, Agile Release Manager, or equivalent in other systems if the teams don’t self-correct. Unlike a sports coach, the job of any business coach is to work their way out of a job. For Full Stack Scrum, as described in the opening pages, I am trying to write a site that will be your coach.
Scrum makes it easy to identify when something is preventing work from moving forward. A “blocker” or “impediment” may be:
- A technical issue that must be resolved for a story or task to be completed.
- A dependency on a group or individual outside of the team that has not been resolved.
- Another story the team must complete first in order to complete the blocked story.
- Lack of progress on tasks other team members must finish for a member to complete his or her tasks within the sprint.
- Events not covered by current-sprint user stories taking time away from sprint work, including:
- Valid reasons such as a customer emergency that could not have been predicted.
- Inappropriate actions the SM will need to address, such as a manager ordering a team member to do non-sprint work.
When a story still in the Product Backlog has a blocker, the team should take actions to remove the blocker, and review it regularly until those actions have done so. Usually it is a very bad idea to plan a blocked story into a sprint. If the blocker does not get cleared in time, the team will fail to deliver that story.
Once a sprint is under way, team members should block early and often to tackle problems while still relatively small. The longer you wait, the less time you have to resolve it and complete the remaining tasks. Blocking is done at the task level in this case, both because it is easier to see where the problem is, and because it is possible for more than one task to be blocked, by different problems. Blocking is done by marking the task as blocked in your tracking tool or on the story card, and by announcing the blocker during the Daily Standup. A Scrum Master must make any blocker his or her top priority once aware of it. The SM might not remove the blocker personally, but making sure it has been addressed and driving a quick solution is a primary responsibility of the role.
Because Scrum drives toward delivering something to a customer every sprint (whether or not that actually happens), the team tries to get a story’s output to a level of quality it would be proud to show the customer. This is supported a number of ways. First, the team is structured cross-functionally. There are no “handoffs” from, for example, the hardware engineers to the firmware engineers, or from the software developers to the testers. The hardware, firmware, software, and test engineers responsible for that part of the product are all on the same team, working on the same user stories. Preferably, the line is blurred between developers and testers: Developers test each others’ work, freeing up testers to add value in other ways. These include exploratory testing—trying to break the product before the customer gets a chance—and test automation, which speeds up delivery significantly by eliminating manual work.
Other FuSS techniques to help the team build quality in from the start include:
- Setting time aside via quality-related tasks in each story to ensure the team does not scrimp on quality to meet the sprint date.
- Not “accepting” stories as completed if defects the team found were not fixed.
- Requiring teams to fix other bugs found during one sprint in the next sprint.
- Recommended use of Agile methods related to quality, such as “Acceptance Test-Driven Development,” which emphasizes writing the test first and then developing to it.
When Primavera Systems switched from waterfall to Agile, specifically the Scrum and Test-Driven Development methods, it saw “a 30 percent increase in quality as measured by the number of customer-reported defects in the first nine months following the release” and a 75% improvement in team-level defect rates.
New versions of hardware and large software systems are impossible to release in a single sprint, in part because usually more than one small team is needed to develop them. Like some other Agile-at-scale systems, FuSS has proven Scrum is perfectly capable of handling multi-team, multi-sprint releases. Relative to others, FuSS makes the process as lean as possible:
- Only the cross-team roles are involved in release planning until requirements are well-groomed, meeting weekly.
- Product Owners are brought in then to:
- Confirm the requirements are clear.
- Predict likely questions from the team and get them answered.
- Represent team wishes regarding which requirements they want to work on.
- Each team breaks its requirements down into user stories.
Note: Done as part of an otherwise normal sprint on the team’s schedule, in one workday or less, this means teams never stop productive development.
- In a final release planning meeting:
- By comparing a team’s story total to its per-sprint delivery rate, the release planners and POs decide whether the team is trying to do more than it can within the release period.
Example: If the team broke its epics into 48 stories, four sprints are left in the release, and the team reliably gets 9 stories accepted each sprint, it is over-committed by more than a sprint’s worth of stories: (9 x 4 = 36) – 48 = –12.
- In that case, requirements or stories get traded to less burdened teams, if possible, or requirements get removed from the release plan if not.
- By comparing a team’s story total to its per-sprint delivery rate, the release planners and POs decide whether the team is trying to do more than it can within the release period.
The combination of the planning process and jointly held demonstration ceremonies eliminates the need for “scrum-of-scrum” meetings. This saves each participant a half-hour or more of time each week with no impact on the system’s predictability or productivity.
In cases where companies or the outputs cannot be fully tested continuously, releases are overlapped. When system validation and regression testing requires a sprint after development teams release their deliverables, for example, the teams go ahead with the next release but allow extra time for defect fixing during the sprint. Whether or not the system testers are embedded in the teams, they create stories for that testing. By overlapping release-related efforts instead of stopping for “big-room planning” or “hardening” sprints, FuSS keeps teams moving forward.
Despite this Lean approach, FuSS principles have allowed programs to hit delivery rates as high as 90% of planned epics in four-month releases.
The work that introduced the term Scrum to business, “The New New Product Development Game,” says that in development efforts it covered, “The project team begins to operate like a start-up company—it takes initiatives and risks, and develops an independent agenda. At some point, the team begins to create its own concept.” The article likens top managers to venture capitalists, quoting one as saying, “‘We open up our purse but keep our mouth closed.’”
At some point when converting a group to Full Stack Scrum, I will surprise them by saying I do not consider myself an Agile expert. Yes, by most standards, I am one. My point is that I came to Agile through my earlier and ongoing interest in leaderless teams. Called “self-directed” or “self-managing” teams when I was introduced to the concept in the 1990s, they have consistently been shown to outperform standard business teams when properly set up.
Ironically, it doesn’t work to act like a teamwork sorcerer and say, “Poof, you’re a self-directed team. Go outperform!” A framework of best practices is required to pull this off efficiently, after which the team can make whatever changes it wants to improve performance. I used my skills from my first career as a journalist to go through more than 600 sources on small groups, mostly scientific studies, to learn those best practices (not to be confused with popular practices “teambuilding” firms sell, which sadly bear little resemblance to what works to create long-term change). I founded a side business, TeamTrainers™ Consulting, through which I trained teams and managers on those practices. Meanwhile I became a technical project manager, using waterfall. By the time I was introduced to Agile in 2008, I had already been studying or working with what the Agile Manifesto calls “self-organizing” teams for 14 years. Seeing these two principles had the biggest impact in converting me to Agile:
- “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”
- “The best architectures, requirements, and designs emerge from self-organizing teams.”
To my mind, employee empowerment was the only concept important enough to the founders to rate two principles! Combined with some other principles, I thought that meant Agile practitioners saw the value of what I was doing in a way many traditionally run companies did not.
Unfortunately, the Agile Industrial Complex has subverted this by convincing companies you need one full-time Scrum Master (SM) for each team. My preference is to have each team rotate the role among interested members. But in most cases I end up training a volunteer with a full-time technical job to do the gig using a few extra hours per sprint. In the exceptions where full-time SMs were preferred, I and every SM I trained easily handled 3–4 teams at once. That means, given the going pay rates for SMs, that companies with three or more full-time, single-team SMs are wasting hundreds of thousands of dollars or euros a year.
“Nature abhors a vacuum,” so when someone is hired for a full-time gig that is not full-time work, they find ways to justify being full-time… and thus become a de facto team leader. When I told a newly formed team of subcontracted programmers on Long Island they would choose their own part-time SM, they got visibly excited. One said, “That’s a real difference, because every Scrum Master I have worked with has acted like they were the team leader.” Everyone nodded their heads. A team I was working with in India two weeks earlier chose to rotate the position, based on the strong opinion of one member who said otherwise that person ends up getting treated as the boss. I found their histories with Agile depressing.
Self-organizing means the team runs itself as a group. The organization has an obligation to provide clear requirements. But no one, including the PO and SM, tells Scrum team members how or when to deliver those requirements. Instead, the members including the PO and SM decide together and tell the organization how and when, on an iterative basis.
Agile advocates generally recommend the creation of teams with stable membership. An in-depth study of Agile team success factors in six Brazilian companies using structured interviews, team document reviews, and direct observation for six months found lower “staff turnover emerged as critical for team productivity.” Although Agile processes like those in this site do much to reduce the negative impact, the researchers found it “surprising that, even adopting such practices, the teams were visibly affected by medium turnover.”
Three professors analyzed data from more than 300 projects over 20 months in one Indian company. This company re-formed teams for each project, and tracked a lot of relevant data like who had worked with whom in the past; success metrics of each project; and various factors that could impact performance. How often project team members had worked together was more strongly linked to team performance than the members’ job experience. What scientists call “team familiarity” was linked to fewer code defects and greater likelihood the project was completed within the labor estimates, or labor and schedule estimates combined. Higher familiarity related to fewer problems in almost every area. So the best predictor of success on most measures was how well people knew each other, not their years on the job or other factors.
Imagine how much more powerful that impact would have been if the company did not break apart the teams! The sad thing is, this information should not be news to anyone with business experience. Check out this quote: “Service troops are trained in working units and will prove most efficient when the unit is kept intact. Men of each squad, for example, usually work best under their own leader, each man knowing from past experience his particular assignment in the group.” This comes from a U.S. Army manual published in 1945!
According to an analysis of data from more than 10,000 teams that used Rally (now CA Agile Central) as reported at the Agile 2014 Conference, “Stable teams result in up to:
- “60% better Productivity.”
- “40% better Predictability.”
Also, “Teams made up of people who only work on that one team have double the Productivity.”
Studies have confirmed each member learns a lot from “past experience” with the group that benefits the group, and takes months or years to re-create in a new team. As I wrote for one client, “In group psychology terms, this is because the following traits of high-performance teams become possible or are enhanced:
- Team cohesion—The sense of identity with a group of people to whom members feel mutually accountable.
- Team potency—Members’ belief in the general effectiveness of the team.
- Team efficacy—Members’ belief in the ability to achieve a specific goal.
- Shared mental models—A common understanding of how work is accomplished.
- Team memory—Understanding of who knows what on a team, and how things worked or didn’t in the past.
- Familiarity—Understanding of each other’s skills, communication styles, etc.
- Trust—Belief that fellow team members will do what they say they will do.”
Don’t move people to the work, most Agile coaches say; move the work to the team. A major objection raised is that the old team may not have the needed skill sets. First I note that the same was true for every waterfall project team I ever managed—they always needed help from some other team, often ones not formally part of the project.
The answer is to let the team figure out how to handle the work for which it does not have the skills. Options I have seen succeed include cross-training with internal experts; formal training from external experts; getting help from other teams; or in the worst case, hiring outside expertise in the form of contract workers. For all these reasons, the creation of stable, full-stack teams is listed as one of the desired agreements for implementing FuSS (page 79).
Agile development teams “usually include members with a broad skill set, such as analysis, programming, design, database architecture and administration, systems engineering, and project management,” wrote three information systems professors. “In fact, agile teams share the premise that everyone who the team needs to fill all the roles necessary to complete the project must participate in the project team. This situation differs from the traditional waterfall/structured environments, where teams often specialize according to function (e.g., analysis, design, development, testing).” The emphasis on cross-functionality also is consistent with studies showing diversity of work background and tenure within an industry improve team outcomes.
These are sometimes referred to as “full stack” teams, a programming term I will redefine as “containing all skill sets necessary for delivering customer-usable functionality, if not entire products or services.” I worked with a company that delivered utility meters, those boxes on the back of your house and office that measure how much electricity you use. Modern versions are usually “smart” meters capable of talking wirelessly to the utility, meaning they have mini-computers and cell phones inside in addition to the “metrology” functions. One of the teams there was rapidly developing the company’s next generation meter. It included hardware, radio frequency, firmware, functional software, interface software, and test engineers. That’s about as “full-stack” as it gets!
The single easiest way to eliminate interface conflicts between functions is to eliminate the interfaces. People writing requirements cannot gripe about “the development teams” not delivering what they asked for, because they are on those teams. Developers cannot “toss over the wall” to testers problematic code, because there is no wall. The testers are on the team. The same is true for each function represented. Another advantage is the impact of each function’s decisions on other functions is understood before they are implemented, a cause of many problems when component teams are used.
Workload balancing and resource planning becomes easier in this setup. Phases in typical waterfall projects tend to use certain skill sets more than others. When people are not available to take one phase’s output, the project is slowed; when the people are available but the output isn’t, the project is slowed and the organization loses money as people are paid for doing nothing. In Agile generally and FuSS specifically, after a relatively short planning cycle to initiate the project, everyone stays fully engaged to the end.
A curious side effect was revealed in the data mentioned in the last section based on thousands of teams. “Interestingly,” the presentation said, “teams that self-identify as having no testers have:
- “The best Productivity.”
- “Almost as good Quality.”
That didn’t mean they did not test. It meant the developers did testing, and testers did developing, and members tested each other, with very positive results.
FuSS allows exceptions to this full-stack concept. For instance, the typical ratio of technical writers to developers means you have to choose between making the writers their own team or splitting each member’s time across teams. In other words, you have to choose between violating stability or violating full-stack capability. From the standpoint of group psychology, err on the side of stability by letting highly specialized functions whose members each support many other teams stand alone in specialist teams. This should obviously be the rare exception, though.
Teamwork studies have consistently shown that “collocated” teams—those able to meet together physically every workday—outperform otherwise similar teams spread across different locations. In the study in Brazil cited earlier, “projects reported significant productivity gains through improvements in communication, teamwork spirit (cohesion), planning, and requirements negotiation when collocated.” There are several reasons for this. The value of face-to-face communication in terms of nonverbals and speed was discussed under “The Principles of Agile” above. Self-organizing requires a high quantity, not just quality, of contact among members as a group and as individuals. Trust cannot be “created,” despite the claim of many teambuilders—it develops organically through repeated positive interactions with another person.
Fortunately, members need not be in the same room, which may make collocation easier for some organizations. In fact, for maximum productivity and job satisfaction, they should not be in “open workspaces” despite the popularity of the concept. See “Does the team need a team room?” on page 392 for the scientific evidence against it.
At the very least, they should sit in the same time zone. That Rally big-data analysis reported: “Teams distributed within the same time zone have up to 25% better productivity.” Certainly they must be close enough to meet regularly as a group, within three time zones of each other at worst. And there is significant evidence it pays off in productivity and reduced conflict to invest in bringing virtual teams together physically at least once a year.
In other words, from the social psychology perspective, there is no such thing as a “global team.” Though I found many proposed ways to overcome the problems of “temporal distance” in the scientific literature on Agile, not a single study provided evidence a solution had been found. Technology cannot fix the “problems” of people needing to sleep at night, and night occurring at different times around the world. Because of that, in widely separated teams, some members are going to miss meetings regularly and/or have less job satisfaction because meetings are held outside normal working hours in their time zone. I refuse based on the science to a call a work unit that does not have the capability of regular, easily scheduled, all-member meetings a “team.” It is just a “work group” of individuals loosely coordinating their output.
Unlike many in the Agile Industrial Complex, I am not going to tell managers in global companies what they want to hear just to sell them on my system. FuSS provides means for teams around the globe to work on the same program, but each of those teams must be composed of members close enough in time zone to practice self-organization through participation in all of the prescribed Scrum ceremonies. Otherwise, you are not “being Agile.”
In a sense, architecture in hardware or software design fills the same role it does in buildings, or in my early career of crafting scenery for plays: It provides the blueprint for how the parts of the deliverable fit together to create the whole. In waterfall, this is supposed to be fully specified before any development teams are formed, so those teams don’t waste labor time figuring out the design or having to change direction radically.
Speaking, I remind you, as a certified waterfall project manager, I am confident in saying system architecture design has often been a primary cause of delays in waterfall projects. Usually one or more issues like these occur:
- The architectural specification takes longer than expected, such that development teams are formed but cannot start serious development.
- Contributing to the above, requirements keep changing after the specification is fairly mature, triggering delays, wasted development time, and/or significant changes in direction.
- The architecture does not fit the needs of the teams, but architects resist changing it—usually for understandable reasons beyond their control, like they have moved on to the next project, or the Change Management System is burdensome.
- Better ways of doing things discovered by the teams are disallowed merely because they do not fit the blueprints.
Recall this Agile principle: “The best architectures, requirements, and designs emerge from self-organizing teams.” This anticipates that architecture will be a joint responsibility of the system architects and teams.
By extension, it will also be developed according to Agile principles, for the same reasons the code or product is:
- The architect will be a full-time member of the program, working closely with the developers throughout each project.
- The architecture will be co-developed iteratively throughout the program by the teams and architect.
- The program team as a whole will have final say over its designs.
- Little time will be wasted on pilots or “proof of concept” deliverables, instead of which the teams will:
- Try one approach to the architecture.
- Keep what works.
- Scrap what doesn’t and try another approach for that.
The expertise of a talented architect prevents teams from wandering long in the wilderness, and reduces conflicts over approach that can delay the product. A certain amount of upfront planning regarding broad technology decisions, design patterns, and components or services is valuable in any project, as pointed out by senior architect James Madison in IEEE Software. “Although much of this is similar to the activities of a nonagile approach, up-front architectural work with agile development includes a subtle but important difference,” he wrote. “The architectural direction should include a range of options rather than a specific solution. A looser set of architectural possibilities is acceptable based on the agile assumption that the empirical knowledge gathered by all participants while building the system will make better options more evident.”
So under Full Stack Scrum, we do ask for an initial plan to get everyone pulling in the same general direction. I have seen this called an “architectural roadmap” by some experts on Agile architecture, but I prefer the term “architectural runway.” Like that for an airplane, a runway provides an initial direction as the plane/project gets up to speed—yet only that much. Once the project is flying, it can turn in whatever direction it needs to go.
Like a airplane’s pilot or navigator, we do keep re-plotting the direction in the form of updated architecture. However, in Agile we only do enough to keep the teams going. Those updates are part of planning the next release or sprint, since our direction may have changed by the release/sprint after that. I call this “just enough, just in time” documentation. Per the Manifesto statement downplaying “comprehensive documentation,” this does not take the form of a formal specification document. Diagrams and bullet lists are the majority of the content in Agile architecture.
The runway needed to start a project can be as minimal as the top level and one level of decomposition. Question marks are not only acceptable; they are expected. The teams and architect will answer those questions together. I do not recommend going past the second level of decomposition before development for the same reason we do not groom user stories more than one iteration at a time. Change is inevitable.
However, the airplane you are flying has to be chosen before you take off! I refer here to the hardware and software to be used in the development work, the “stack.” Madison’s warning is on point: “Plan carefully; hardware and software changes are inherently nonagile.” In both waterfall and Agile projects, I would expect a good architect to consult with the developers on making those decisions, though in established companies most of these are “pre-made” in the sense that you are going to re-use existing resources. Regardless, include these decisions in the runway documentation.
A Boeing 747 needs a longer runway than a single-engine propeller plane, and the same is true for projects to build a Boeing 747. Much more of a hardware product must be designed before you spend money on a tangible prototype. Even plastic mockups can be expensive, despite 3D printing! Often, too, a company wants a preliminary bill of materials (BOM) specifying likely parts and their costs before approving a hardware project.
The same Agile architecture principles apply, however. The iterations just take longer:
- Write requirements and stories for specific parts of the architecture, creating it iteratively.
- Conduct reviews with the engineers who will use the architecture documentation each sprint, or at least engineers with similar skill sets.
Note: Those folks can still have most of their time dedicated to other projects, but treat them like a “customer focus group” to increases the likelihood of meeting the chosen group’s needs.
- Incorporate their feedback into your next iteration.
- Start the project team as soon as those folks say the architecture is mature enough, instead of waiting for it to be “finished.”
- Co-develop the design from that point forward.
This approach, like the rest of this chapter, lays out a huge amount of change for many organizations. Making these changes will cost a lot of money in the form of people’s time, not to mention frustration or consternation from many. No valid business decision is made solely on the basis of costs, however. We must compare those to the potential benefits as we “Decide to Change.”
As mentioned, in the Planning Ceremony on Day 1 of the sprint, you will create a list of stories you expect to complete in the sprint. In FuSS, a Sprint Plan is not a wish list of the stories you hope to accomplish over the next few weeks. It is a promise. The verbal contract you make with your customer(s) is this: “If you agree to wait until the end of the sprint to change our workload, we agree to deliver that workload almost every sprint.” My training for Agile certification called this “commitment-driven iteration planning.” In marketing terms, I would call it “under-promise and over-deliver,” a simple principle for making your customers happy.
Customers and internal stakeholders want to know what is coming in the near term because there are things they need to do to prepare. In a retail store, it takes time to update the cash register and inventory systems, set aside space on the shelves, train sales associates, and prepare announcements of the new product. When a vendor gives the store a date for an item, and doesn’t deliver that exact item on time, you can cause them financial harm, not to mention embarrassment.
The same is true if you deliver that item and the customer finds something wrong with it. That retail store starts getting returns from angry customers. Not only does the store incur the costs of returning those goods, its reputation is harmed. The teenager who bought a shirt that came apart at the seam on the first wearing doesn’t care that the clothing store didn’t make the shirt. He or she is going to be mad at the store. After repeated instances of poor quality, people will stop going to that store. In turn, the store is going to stop buying your goods.
With a little imagination, you can create an allegory between a retail store and your company, for example by thinking of the product manager as the store manager and his customers as the shoppers visiting the store.
All this may seem a strange analogy when you hear that FuSS never promises a specific delivery for a longer period than a sprint, for the simple reasons that humans cannot predict the future. It has achieved four-month predictability as high as 90%. More importantly, the method refuses to lie to the customer about what it can predict. A combination of higher transparency and higher customer involvement ensures the “store” and its customers aren’t taken by surprise by either a delivery date or poor quality.
Project management is an organized attempt to help customers by meeting clearly defined “customer expectations” about what the customer will get, by when, in a customer-pleasing condition. The problem with waterfall is it doesn’t easily update those expectations. Full Stack Scrum guarantees you will meet or beat those expectations most of the time, even when the delivery date is a few months away, by proactively managing the expectations.
There is plenty of hard evidence on the Web and in books proving the measurable value of Agile to companies and most employees, some of which I have cited already. These include significant bottom-line financial benefits. I won’t bother repeating them here, since you can easily look them up—and probably already have, since you are (still) reading this site!
In general, Agile and Scrum capture the reality of research and development work better than waterfall. I have personally witnessed a number of benefits from Scrum since my introduction to it at a small Agile startup in 2008. Specifics include:
- Quicker delivery of value to the customer—Instead of having to wait months to see some return for their money, the customer can see requirements move from discussion to reality in a matter of weeks.
- The advantages of self-management—As noted earlier, a huge amount of research going back decades shows that teams with no daily leadership outperform similar managed teams, producing more and better work at lower costs and stress. FuSS efficiently maximizes self-organization.
- A team structure—That same body of research proves that teams taking time to structure themselves by defining roles clearly, identifying team rules and procedures, documenting processes, and planning their work are more effective than those who “wing it.” Scrum gets the team most of the way to an effective structure in a few weeks.
- Better decision-making—Also clear from those studies is that cross-functional teams make better decisions on average over time than a single expert on the given topic. Agile hinges on this kind of decision-making while providing boundaries to prevent it from bogging down.
- Acceptance of change—People naturally resist changes in direction to their work. Agile addresses this by timeboxing change into small increments, balancing the financial and emotional costs of frequent changes against the reality that requirements do change. It also shows benefits to the individual relatively quickly compared to other, less systematic approaches to process change, reducing ongoing resistance.
- Higher mid-term predictability—Most traditional projects fail to meet their schedule, budget, scope and/or quality commitments. By focusing on consistent rather than maximum delivery, FuSS teams arrive at a point where they can promise delivery of all or most stories every sprint, and thus more accurately forecast deliveries over several sprints than a waterfall plan done months earlier.
- Reduced team stress—Because workload commitments are based on historical data, teams stop promising to do more than they realistically can. They also get hard data to support limiting those commitments, or to prove the need for cross-training and/or increasing the number of people on the team. This reduces chronic overtime with its well-documented personal and financial costs, and eases pressure to do more without more resources.
- Reduced manager stress—Managers reduce their stress, too, because they get better visibility into team progress yet no longer have to manage the team on a daily basis. Plus, their forecasts to customers and other stakeholders become far more accurate and they, too, get hard data to support those forecasts.
- Better value—Each story is developed to the customer’s acceptance criteria, ensuring it meets their needs.
- Better quality—Since quality is emphasized at each step and complete testing is part of the delivery, the chance of “escaped bugs” getting through to the customer or end users is greatly reduced. This in turn increases customer satisfaction and lowers rework costs.
- Higher job satisfaction—Drawing on research about the factors that increase job satisfaction going back to the 1980s, three scientists surveyed software developers and found the use of Agile project management practices was strongly correlated with job satisfaction, primarily due to the sense of job autonomy and regular feedback on performance. A scientific survey of more than 1,000 people within Nokia after it converted to Agile found that only 10% would go back to waterfall given the option.
This graphic summarize the benefits for most people from their organization’s switch to Agile:
One financial benefit to the company the Agile Industrial Complex tries to hide is that fewer manager roles are needed, hence the “almost” in the section title. This has always been a barrier to the adoption of self-organizing teams, which is why most consultants won’t discuss it openly. I will shortly, including ways the people in those roles may still be able to add value.
I may make some of you nervous with my final “benefit,” but bear with me. I am not injecting my religious beliefs into this discussion. Rather, I would like to raise a sociological argument. Who said this?:
Do not do to others what you would not have them do to you.
People raised in Christianity or Islam might believe Jesus was the originator of this saying. But notice the word “not” that appears twice. That word does not appear in the quotation in The Bible. The Chinese philosopher Confucius wrote that line around 600 years before Jesus was born. However, another version of it appears in writings dating much further back. Sometime after Confucius quoted it in his Analects, it appeared in a Hindu scripture in India. In 70 B.C.E, a Jewish writer says it. Then Jesus. Somewhere along the lines it lost the “nots,” but of course the meaning is the same.
The fact this line that some call the “Golden Rule” shows up in so many different cultures suggests it is a rule every human will agree on, even if we debate the specifics of how to apply it. When I was young I embarked on a study of the world’s religions and philosophies, and I constructed a list of behavioral guidelines that were common to every one of them. I think all of those can be tied back to the “Golden Rule.”
One can quibble with the rule—some troubled people hurt themselves, so the rule when strictly applied by them would mean they could hurt others. Certainly people apply the rule differently, because we disagree about what is right and wrong to do to ourselves and others. Yet I think when we get upset with someone, the root cause is our belief that the other person did something violating the Golden Rule. “I would not do that to them,” we feel, “so they should not have done that to me.”
Think about something physical you buy online or over the phone that is important to you: electronics; clothes; in my case, sailboat parts. Whenever you order that thing, you have to choose a shipping method. Those methods always come with a range, like “5 to 7 business days.” Unless you’re rich or a shopaholic, you debate among several different items you can get with the limited amount of money you are able to spend right now. You make your choice, and get excited that within a week, it will show up.
If you have been paying attention over the years, you have noticed that most of the time, it shows up toward the front end of that range. Almost everything I expect to receive within “5 to 7 business days” shows up in four or five. Now what happens if you get to Day 8, and the thing hasn’t shown up yet?
You get upset, right? Even if you are a patient person, you notice the problem, and if the delay continues, you eventually are on the phone demanding to know where your item is. When people give you a date by which you can expect something, you get upset if it doesn’t show up by that date.
Whether on time or not, let’s say you eagerly open the box and find… something other than what you ordered. Or it’s the wrong size. Or it’s broken. Or it doesn’t work the way the Web site said it would. You get upset, right?
You know where I am going with this. With every Sprint Plan, you tell your stakeholders that you are going to deliver these five or 10 items within 10 or 15 or 20 business days, and here’s how each will work, and they will work correctly. When you don’t deliver 100% of those items, and any of them have problems (defects) that the customer notices, you are treating the customer the way you don’t want to be treated when you are a customer. You are violating an arguably universal rule of fair treatment.
 Naylor, J.B., M. Naim & D. Berry 1999, quoting: G.H. Watson (1994), Business Systems Engineering: Managing Breakthrough Changes for Productivity and Profit, Wiley:New York.
 AMA 2007.
 Curtis, Abratt, Rhoades & Dion 2011.
 Wheelan 1994; Hackman & Katz 2010.
 Morgan 1995.
 Schwaber & Beedle 2002.
 Project Management Institute 2013.
 Takeuchi & Nonaka 1986.
 van Assen 2000.
 Baramichai, Zimmers & Marangos 2007.
 Cooper 2016.
 Cooper 2016.
 van Ruler 2014.
 Christy 2016.
 Schwaber & Beedle 2002.
 You may see a cartoon in Agile sources referring to participants and observers as “pigs” and “chickens.” I avoid the terms because people from some cultures find it insulting to be called a pig.
 Screen shot from the tool now called CA Agile Central, then known as “Rally.”
 James 2016.
 Schatz & Abdelshafi 2005.
 Takeuchi & Nonaka 1986.
 Melo, Cruzes, Kon & Conradi 2013.
 Huckman, Staats & Upton 2009.
 FM10-22, War Department Field Manual, October 1945, titled, “Quartermaster Depot Company, Supply.”
 Maccherone 2014.
 Morgan 2016.
 Tripp, Riemenschneider & Thatcher 2016.
 Maccherone 2014.
 Melo, Cruzes, Kon & Conradi 2013.
 Maccherone 2014.
 Madison 2010.
 Tripp, Riemenschneider & Thatcher 2016.
 Laanti, Salo & Abrahamsson 2011; another 30% saw no difference, leaving 60% who did see a difference and would not go back to waterfall.
 Waley 1938.