Designing an Accountable Software Development Organization Part 3: Roles, Organization, and Culture

In part 2 of “Designing an Accountable Software Development Organization,” I discussed creating a vision for the software development organization that serves as the desired end-state. I explained that levers and tensions exist between the organizational components of the end-state model as well as how they could be harnessed to drive stakeholder behavior in healthy directions. In this third and final article. I will show how to align the roles and organization of a software development organization with the levers previously discussed. Finally, I will enumerate a set of behavioral guideline that help to keep healthy tensions healthy while avoiding unhealthy tensions.

Roles and Organization

As a software engineer by training and education, I have always believed in a pragmatic “form follows function” approach. With few exceptions, I’ve been fairly successful when applying it to organizational dynamics or software development organizations. My goal then is to define roles and align organizations along the natural lines of the levers I previously defined. This way, I assign authority and responsibility that is inherent within the vertical domains directly to directors and their organizations. This alignment encapsulates authority and responsibilities within the discrete organization and assigns leadership to a single director who manages his/her team, negotiates healthy tension-related issues with his/her peers, and provides me with a single point of contact to report on the status of the lever under their care.

In my software development organization, I have defined three key organizations: product management, technical delivery, and operations. The roles flow from the responsibilities and authorities defined earlier:

roles

I have made a few accommodations to a straight one-to-one alignment based upon the mission and skill sets. First, because my mission is to create a next generation product while maintaining and supporting the existing product, I have created two technical delivery teams. I found it prudent to ring-fence development staff within product domains to avoid having staff distracted by issues outside of their respective domain. For example, when I have had product teams intermingled, there was a temptation to divert key next generation product staff to solve knotty problems within current product domain. All too often, this proved a distraction to the mission of developing the newer product that I sought to avoid. Second, since the roles of risk tracking and process adherence are closely related without having inherent tensions between them, I collapsed both roles into an “Operations” domain and leverage the project managers in both roles.

Culture and Ethos

Now that we have defined the roles and organization along the lines of the levers, it’s time to turn to the management of tensions. The essence of developing great software lies with balancing the various forces at work within and between the organizational components. I have designed my organization such that healthy tensions bubble up to the director-level within each organizational role while minimizing the occurrence of overlapping unhealthy tensions. However, where there is tension, there is the opportunity for conflict. In keeping with my goals of engendering communication and trust, I have a simple set of rules for dealing with this that focuses on rules of engagement as well as encouraging professional maturity and development.

As described in the levers and tensions section, I make the responsibilities and authorities of each role explicit. I take a “carrot and stick” approach to describing the behaviors I expect and those that I wish to avoid. I set clear expectations for the directors as in the following example:

ethos1

Setting clear expectations about authority and responsibility boundaries is frequently reinforced within the organization. The directors understand this and set development goals for their staff accordingly.

When inevitable conflicts do occur, I prefer to set expectations for appropriate means of resolving them. At the risk of sounding “preachy”, our business partners and peers expect and deserve to be engaged in a respectful and dignified manner even when disagreements arise. If a software development organization is to communicate effectively and engender trust, I believe that a high level of personal integrity must be maintained in interpersonal relationships. If we don’t behave appropriately, then we have little right to expect trust and respect in return. We have all seen recent reports where a major automobile manufacturer was caught cheating on emissions tests after marketing their engines as clean. This was a clear example of where a technical delivery organization broke the trust with their consumers by acting with low integrity. Consumers are correct to no longer trust their claims.

I provide a simple set of guidelines for engaging staff, peers, and their managers. I again mention behaviors I expect that those I wish to avoid:

ethos2

I think most of these speak for themselves. I would like to comment on “taking the high road” a bit further. There are political issues and rivalries in any organization. Frequently, it is easy to take the approach of “fighting fire with fire.” I reject this approach for the simple reason that while such an approach might be emotionally satisfying, it will almost inevitably lead to regrets. Regret, whether in personal or professional affairs can be emotionally corrosive and ultimately self-defeating. I try to avoid situations leading to regret whenever possible.

Getting Results

So what results can you expect from a truly accountable software development organization? In my most recent experience, the software development teams saw positive quantitative and qualitative results over a three-year period. The teams’ on-time delivery went from hitting the mark less than 60% of the time to achieving on-time delivery over 90% of the time. The software quality as measured by post-delivery fixes increased by over 75%. While we applied the Agile Scrum methodology to an inherently risk-averse business domain, we have consistently passed client audits with only minor findings.

Qualitatively, our business partners report that they are more consistently satisfied with the software solution. This increased satisfaction has led to increased confidence in the software development organization. The software organization itself has developed a new confidence and sense of satisfaction knowing that they deliver high-quality solutions when the business needs them.

My experience is limited to software development and infrastructure technologies and I have applied the above techniques within those domains. However, the principles of empowerment and alignment should apply to any business domain. I would love to hear back from those who try their hand at the application of these principles!

Best, Charlie

Designing an Accountable Software Development Organization Part 2: Vision, Levers, and Tensions

In part 1 of “Designing an Accountable Software Development Organization,” I laid the foundation for software development accountability by translating the business needs required from the software development organization into a clear statement of purpose. I went on to define tenets and principles to support the mission and underpin its implementation. In this article, I will discuss creating a vision for the software development organization that serves as the desired end-state. Levers and tensions that exist between the organizational components of the end-state model will be covered as well as how they can be harnessed to drive stakeholder behavior in healthy directions. Finally, since behaviors can become unhealthy, I will discuss the source of such behaviors.

Vision

Other than in cases of natural staff attrition, new software development leadership is installed when the software development organization has not met the goals of the business that it is entrusted to serve. The new leadership has an opportunity to reset the organization to meet those needs. In almost all cases and after some reconnaissance, the new leadership defines a new organizational end-state. That end-state is intended accomplish specific goals. First, the end-state serves to operationalize the mission of the software development organization. Second, it points to realistic expectations and outcomes for the organization. Third, it provides a baseline for a regimen of continuous improvement. Finally, it helps establish a clear line of sight to the business goals and helps each member of software development organization understand how they contribute to those goals. In keeping with the principle of subsidiarity I discussed in the previous article, I recommend that you share the development of the end-state with your software development organization as it’s developed. Doing so both enlists the staff’s support in realizing the vision and gives them a chance to add criticism and correction in cases where you might fail to understand nuances in the existing culture or personalities.

In the example I used in the part 1, I was hired to shape a historically consulting-based software development organization into a product-centric and customer-focused one. The primary goal was to create and deliver new high-quality, innovative solutions predictably to the business while maintaining and enhancing existing products. I will add that this was in the domain of software supporting pharmaceutical clinical trials. For those who aren’t familiar this business domain, patient safety is of paramount concern. Because of this, it is among the most highly regulated and audited sectors of the software development space. Consumers of the software tend to be highly conservative and risk-averse. For example, customers routinely audit the software and the SDLC to ensure complete traceability between requirements and tests as well as to ensure that the SDLC is sound and followed meticulously. (As an aside, we implemented an Agile Scrum-based SDLC discussed in the article, ”Agile/Scrum Adoption in Regulated Industries.”) To address the patient safety concerns, we needed to produce exceptionally high-quality software that was well fit to the business need. To address the regulatory needs, we needed a strong focus on the software development lifecycle and its governance. We also needed to continue to support the existing product suite even as we developed its successor. It was clear that I needed strong alignment of authority and responsibility within the requirements, technical delivery, and SDLC governance domains. The organizational end-state was as follows:

vision

The gray boxes to the left represent the sources of business requirements feeding the product pipeline. The arrows at the top represent the product development process including product management and development steps while the boxes below represent the organizational landscape needed to support the process. In short, this depicts the end-state for the software development organization. It also helps underscore the strategic goal of reliably delivering innovation while pointing to the organization needed to realize the tactical implementation.

Levers

The leaders of a software development organization constantly adjust the direction of the organization as the needs of the business change and as risks arise. They do so by exerting leverage on key areas of influence. In the case of the organization I was designing, I identified 4 “levers.” Requirements fidelity refers to the fitness for the business purpose and prioritization of the feature pipeline. Technical delivery refers to the development capacity, timelines, architecture, and quality. Process adherence refers to SDLC governance including audit fitness. Risk tracking refers to the reporting of project status and risks. The responsibility and authority within the domains are as follows:

levers

As you can see, it was crucially important that the authority and responsibility aligned vertically within each domain. This vertical alignment ensures that each domain can act independently within their area of responsibility and drive efficient execution. The domains intersect at areas where healthy tension is required to balance the responsibilities of the domain. Those intersection points are precisely the leverage points where influence is best exerted. You will also notice that the intersection points are also places where the most discord can occur within a software development organization. These intersections were placed purposefully to ensure that healthy tensions rose to the level of the technology leadership visibility rather than being hidden from them.

Tensions

I spoke of healthy tensions in the last section. In my organizational design, I wanted requirements fidelity, technology delivery, risk tracking, and process adherence to act as independently as possible while having the freedom to report risks without fear of consequence. For example, those responsible for the requirements have the goal of ensuring the product meets the needs of the customer. If some requirements complexity made it harder to develop technically, then I wanted that tension to be explicit so it could be adjudicated at the right level and by the right people. By the same token, insufficiently detailed requirements could create risks to timely technical delivery that needed similar adjudication.

I say these are “healthy” tensions because balancing them is necessary for the development of great software. However, I have seen organizations where the authority and responsibility overlap to create unhealthy tensions. Here are a few examples:

tensions

In the above example, I have taken the four levers and shown what can happen when inappropriate superiority/subordination occurs between the domains. For example, I once inherited a technical delivery team where the project manager reported to the technical delivery management. In some cases, projects were delivered late. I discovered that that project manager did not feel free to report risks to the delivery timeline leaving those risks largely hidden. Clearly, the project manager felt that there could be consequences if risks were reported that painted the technical delivery team in a bad light. Leaving these risks unaddressed was frequently the source of late technical delivery. Conversely, I have seen organizations where the project managers had the authority to set delivery timelines on behalf of technical delivery teams solely based upon when the business wanted the software delivered. It was no accident that the technical delivery teams seldom met the timelines since they were never given the opportunity to produce realistic estimates. In both of these examples, the morale of the staff suffered because they felt, correctly I might add, that they lacked the authority to execute on their responsibility to the business. The project manager in the first example and the technical delivery team in the second felt victimized. The remainder of the unhealthy tensions speak for themselves.

When designing your organization, it is important that you align the vision for the organizational end-state with the mission of the software development organization. The organizational components should balance vertical authority and responsibility with the requirement to make healthy tensions explicit. Care should be taken to avoid authority and responsibility overlaps that create unhealthy tensions that could sabotage meeting responsibilities and produce low morale. It can be a delicate balancing act. But when done with a deft hand, it can ensure that execution is efficient, and morale remains high since the teams have the proper authority to ensure that they can meet their responsibilities. Accountability is the natural outcome of a successful balance of authority, responsibility, and adjudication of healthy tensions. As each domain delivers on their responsibility, their confidence in their delivery capabilities grows. High morale for the software development organization and trust from their business partners are natural side effects of true accountability.

Preview of Part 3

In part 3 of this series, I will discuss aligning roles and responsibilities along the lines of the organizational levers. I will also describe engagement guidelines that can create a culture or “ethos” intended to keep healthy tensions healthy while limiting the expression of unhealthy tensions.

Stay tuned!

Designing an Accountable Software Development Organization Part 1: Tenets, Mission, and Principles

All too often the word “accountability” is associated with the word “blame” when used in the context of software development organizations. In these situations, the software development organization was viewed as somehow failing to meet the business need. Either the software solution came in late, or it contained insufficient features or quality to pass muster. Sometimes the software was seen to fall short on each dimension. When examining the causes of these failures, I have found that they usually result from three broad categories of issues. The mission of the organization was unclear, the technology leadership did not align the software development organization for success, or those same leaders did not empower their staff to be successful. Frequently, all three were the case.

In my previous article, “The Executive and the Software Craftsman,” I discussed how technology leaders can prepare their software craftsmen to make better business decisions. In my subsequent article, “The Technologist’s Guide to The C-Suite,” I went on to explain what the business expects from software craftsmen and other IT professionals. In both cases, it was important that the business and technology leadership built and maintained a strong relationship with their technology delivery staff based upon communication and trust. In a series of three articles, I will lay out a framework for harnessing that trust into an accountable software development organization by accomplishing three key goals:

  • Clearly enumerating a mission for the software development organization to ensure that all parts of the organization are pulling in the same direction. Basic tenets and principles must be spelled out that support the mission and underpin implementation.
  • Creating a coherent vision for the software development organization that defines the desired end-state. Flowing from the vision are levers and accompanying tensions that must be placed in proper alignment and managed to reach the desired end-state.
  • Aligning roles, culture, and ethos to help keep the tensions healthy while continuing to drive the organization in the right direction.

Tenets

As with many other technology executives, I have had the benefit of working with and for some talented managers. Along the way, I have synthesized some basic tenets that guide my organizational design and development. These include:

  • In a principle called “Subsidiarity” problems are best dealt with at the most immediate or local level consistent with their solution. L. David Marquet describes this as locating authority close to the information needed to make decisions in his talk on Greatness. In a software development organization, this requires that the staff is armed with the best information to make good business decisions. Establishing line-of-sight to the business problems to be solved is critical to arming people with the best information to make decisions.
  • Healthy tension exists between departments and when properly aligned, can provide checks and balances to yield better outcomes. The tensions must be identified, explicitly called out, and managed, so they don’t become unhealthy. The assumption is that tension is a healthy and necessary part of the software development process. Misalignment or mismanagement of healthy tensions can quickly turn healthy tensions into unhealthy ones.
  • Clarity of purpose within the organization, alignment of responsibility and authority with a departmental domain, and management and adjudication of healthy tensions between departmental domains are the antecedents of accountability.
  • Good and bad culture (Ethos) is no accident and is a consequence of setting expectations for and management of healthy and unhealthy tension, or a failure to do so.

Mission

I’m sure most readers helped develop departmental or corporate mission statements and had rolled their eyes along the way. I’m no exception. All too often such statements are developed by committee and end up being watered down or so esoteric as to be meaningless. However, when done right, such statements can be essential to providing valuable context and clarity needed to align a software development organization with the business domain it serves. I have developed some simple questions and rules for developing mission statements for software development organizations that I have found helpful:

  • How does your software development organization specifically contribute to the business?
  • The mission should inform both organizational strategy and tactics.
  • Five or fewer bullet points should be sufficient to capture the essence of the mission.
  • The mission statement should be a reflection of the technology executives character and not reflect group-think.
  • The mission should stand the test of time and not significantly change from year to year.
  • The mission should be reinforced repetitively with the staff at every opportunity.

As a concrete example, I was hired at my most recent job to shape a historically consulting-based software development organization into a product-centric and customer-focused one. The primary goal was to create and deliver high-quality, innovative solutions predictably to the business. After two months of reconnaissance, I developed the following:

mission

This mission has not changed in close to three years and continues to provide clarity to the department. I also reiterate it at every opportunity, especially during departmental briefings and strategy/goal planning sessions.

Principles

Just as I have development tenets that guide and inform my organizational design and development, I have developed principles that guide the development of great software. I use these to help my staff understand “what makes me tick” and help them understand what I expect from them and how I make decisions. As such, I humorously call them “Charlie-isms.”I like to think that some of these principles have helped in the career development of my staff. These are:

principles

I will not delve deeply into all of these, but two are particularly near and dear to me. Since I was a developer myself for many years, I have a running joke that goes, “Software developers have two jobs: Fixing bugs and creating bugs.” When we consider this seriously, unfit solutions and low-quality code sap the development capacity of a software development organization. A persuasive business case for solution and code quality beyond the obvious ramifications emerge. Namely, that the production of quality software means that you can devote more development capacity to developing more great software rather than fixing what you’ve previously built. Hence, “Product quality increases development capacity.” There are sometimes valid reasons for taking quality shortcuts especially when timelines are critical. However, delivering low-quality software is seldom beneficial in the long-term.

As a second point, development organizations frequently feel besieged by their business partners when the software development organization does not meet the goals of the business. When this happens, the confidence of the software development organization can erode and lead to a culture of negativity within the organization that manifests itself as “no, but” responses to requests from the business. For example, when a software development organization is asked if they can deliver a given software component within a fixed time, the response will frequently start with a “no.” “No” is then followed by the enumeration of the conditions that will lead to a successful outcome. By the time the discussion turns to meeting the request, the people making the request have already decided that the software organization is an obstacle to progress rather than a partner to success. I counsel software development organizations to understand that most requests are reasonable given the right combination of requirements, time, and staff to deliver the solution. When engaging the business, I turn the conversation into one where the software development organization discusses costs of delivering the solution to meet the goals of the business. Once the business understands the costs of meeting the request, a “yes we can” rather than “no we can’t” tone becomes apparent. The business then views the software development organization as a partner committed to understanding and meeting common goals.

Preview of Part 2

In the next article, I will discuss creating a vision for the software development organization that serves as the desired end-state. Levers and tensions that exist between the organizational components of the end-state model will be covered as well as how they can be harnessed to drive stakeholder behavior in healthy directions. Finally, since behaviors can become unhealthy, I will discuss what can be done to prevent that from happening.

Stay tuned!