The Power of Intention (in software development)

In my previous article, “Designing an Accountable Software Development Organization Part 2: Vision, Levers, and Tensions,” I discuss the tension that exists between requirements, governance, and technical delivery activities within a software development organization. The tension between these activities is a necessary part of developing truly great software since they provide checks and balances between sometimes competing agendas. In that article, I illustrate the effects that tension on misaligned organizations and discuss how better organizational alignment helps the software development leader manage those tensions and keep them healthy.

Even after achieving the best organizational alignment which allows tensions to bubble up to the right level where they can be managed, the requirements, governance, and technical delivery folks will occasionally go toe-to-toe over issues that arise. The question is when things get heated between disagreeing parties, how do we ensure that tensions stay healthy or more importantly, how do we restore health to an already unhealthy tension? The answer is both simple and complex and comes down to a single word: Intention.

The Right-Wrong Game

The need to be right and to avoid being wrong is hardwired into people. Unhealthy conflict is defined in terms of what I call right-wrong games. Right-wrong games are usually characterized by zero-sum thinking where one party believes that to be right; the other person has to be wrong and vice versa. Also by definition, there is no middle ground and no giving ground. When I see people engaging in these conflicts whether, through email, text messages, or other means, I imagine them sitting opposite one another at a table doing verbal battle face-to-face. When you dig deeper into these conflicts, there is usually a single root cause: each party has attributed a contrary intention to the other party and is allowing that preconception to drive their actions and emotions.

conflict

Let me give you an example. A product manager and software architect are locked in a battle over software requirements and how to implement them. The product manager insists that the requirements must be accepted in their entirety in order to meet the customer’s needs while the architect is looking for requirements flexibility in order to make a more maintainable implementation. The product manager believes they have the customer’s best interest at heart because requirements are paramount and that the architect wants to take shortcuts to make their job easier. The architect believes that making a more maintainable solution will benefit the client by ensuring higher quality and that the product manager is taking a short term view and being inflexible.

So what’s happened to make the relationship fail? The product manager and architect have attributed bad intentions to their opponent and then allowed their perception to drive behavior and emotion. In short, each party believes that that other is out to thwart their best intentions, and each party is hell-bent on stopping the other because each party is convinced of the virtue of their position.

Breaking the Right-Wrong Game

When faced with these conflicts, the best outcome is to get both people on the same side argument working toward a mutually beneficial solution. In short, they end up side by side on the same side of the table looking out together toward a common solution.

cooperation

So, how is this accomplished? The parties in conflict need to come to the understanding that they both share the same intention. Namely, to provide the best solution for the customer. They will come to realize that they both want the same outcome and only differ on the means to achieving that outcome. They must realign on their intention and acknowledge that the other party shares that intention. This does not usually happen in the heat of a right-wrong battle and requires a cooling down period first. Even then one party or the other may not what to give ground. It’s the job of the software development leader to bring the warring parties together and get them to reveal their own intentions and acknowledge the good intentions of the other person. With a little guidance from the software development leader as the a third-party, the two parties will come to realize that they both have the same intention at heart. The goal is for two parties to become allies in finding a solution that aligns on their common intention.

Reducing the Occurrence of Right-Wrong Games

The key to reducing the occurrence of conflict in software development organizations caused by intention misalignment issues is to prevent such conflicts from starting in the first place.  To do this, the software development leader must define the intention in collaboration with their team, seek alignment on the intention, and constantly reinforce it. In my own software development organization, we have three primary intentions: Delivery software that is the best fit for the business, deliver high-quality software, and deliver it when we promise we will. These three intentions may seem intuitively obvious for a software development organization but, often, they are assumed rather than made explicit and are spoken about in a trite manner.

Recently, our team delivered a major software release designed to allow more customers to migrate to our new platform. The team pushed extra hard to meet a deadline and succeeded in delivering on time despite numerous obstacles. At the weekly status meeting near software delivery time, I made sure to reinforce with the team how they supported the business by delivering the right functionality, with high quality, and on time. All throughout the effort, the team was focused on the goals and aligned on our common intentions. When obstacles occurred, they came together and found ways to overcome them…usually on their own and without management intervention. They made a hard task look easy all because they worked together. I couldn’t have been prouder of the team for their commitment and the results.

Be Easy on Yourself

As a software development leader, it’s all too easy to get frustrated and annoyed when inter-team conflicts arise. I know this first-hand because I sometimes feel this myself. When I have a moment to step back and reflect, I remember that software development is the art of balancing tensions to get the best results. Software development is a human endeavor involving high-performing people who sometimes see their peers as obstacles to meeting their intentions. When you want to jump out a window out of frustration born of interpersonal conflict caused by misattribution of intention, remind yourself that your software development staff is well-intentioned and wants to do the right thing whenever possible. In short, be easy on yourself for not being able to avoid these conflicts and be easy on your team since we all share the same virtues and vices when it comes to relationships. Your team needs a periodic and gentle reminder that they all aligned with the same intentions. Rather than forcing a result, more likely than not,  you will find that your team will come together and find their own solutions and get a sense of satisfaction in doing so.

Best,

Charlie

Mining for Diagnosticians

There are many diverse talents among technologists. Some excel when challenged with creating new applications. Others love to maintain and enhance existing systems. Others thrive when developing system-level tools and utilities. Still others gain validation from designing and implementing exceptional user experiences. One of the most intriguing and compelling skill sets is that of the diagnostician.

Complex and vexing problems arise throughout software development projects. Developers are constantly diagnosing and solving these problems which make the diagnostic skill set part of every developer’s repertoire to varying degrees. Having been exposed to many software projects and many developers, I’ve noticed that diagnostic abilities are a key discriminator between good developers and great developers. Further, I’ve also noticed that the more exceptional diagnosticians often eventually find themselves in leadership roles. I have asked myself what distinguishes the skills of a great diagnostician and what these skills might have to do with career progression. I have come up with some key factors that I believe are implicated in the phenomenon:

Holistic View

The best diagnosticians maintain a broad view of the systems they develop. They may be experts in some parts of the system, but they understand the coupling between components and the flow of information between them. They are able to understand the system at multiple levels from logical design to physical implementation. This holistic view represents a mental map of the system and helps the diagnostician to navigate rapidly through its parts. This ability to create and maintain a big picture view is also commonly found in those who end up leading technology endeavors.

Deep Bag of Tricks

Complex problems frequently require complex solutions. Master diagnosticians maintain a deep and varied set of tools to help them explore problems. I have witnessed developers abandoning their canned tools altogether and resort to lowest common denominators techniques akin to print statements and traps when the most advanced tools give up the ghost. Innate resourcefulness and a willingness to try anything are traits found in many leaders.

Avoids Wild Goose Chases

Getting stuck on a problem is a common occurrence when diagnosing a software problem. The best diagnosticians seem to have an intuition about when they are traveling down a fruitless path. They actively and quickly eliminate fruitless paths and move onto other opportunities. Put another way; they focus on the goal of solving the problem and follow problems wherever they lead. The best leaders intuitively temper dogmatism with pragmatism to find more efficient paths to goals.

Values Accuracy over Guesswork

Making assumptions and testing hypotheses are part and parcel of the diagnostician’s trade. However, great diagnosticians temper guesswork with accurate assessments of their findings. How many times have you seen a developer insist that the source of the problem must be XYZ only to find out later that they were far off the mark? The best diagnosticians spend very little time being attached to their assumptions. They accept the evidence and move on. Leadership also requires testing assumptions based upon data and changing course when the evidence does not fit the assumptions.

Keeps Their Ego at Bay

It’s human to believe that our own efforts are probably not the cause of problems. In software development, it’s all too easy to assume that the problem is caused by someone else’s code. The best diagnosticians often assume that their software could very well be the cause of the problem and will look there first. They park their egos at the door in their search for the answer leaving no stone unturned. Leaders that are best at empowering teams frequently exhibit this same ego-less approach to social interactions. They are not afraid to admit mistakes and change course as a result.

Avoids the Blame Game

It’s all too common too for technical staff to point the finger at each other when difficult problems emerge. Master diagnosticians avoid this game altogether. They know that blame is pointless and harmful to the diagnostic relationship where collaboration is the key to efficiently solving the problem. Great leaders also focus on moving disparate teams toward a goal and know that cohesion is more effective than division.

I’ve made it my business to keep on the lookout for great diagnosticians. Frequently, these are the people who make the best leaders since they understand the overall structure of the system, know how to explore paths to solutions efficiently, value accuracy, and seek to operate with low egos and high collaboration. I see the best diagnosticians as “diamonds in the rough” when considering people for positions of technical leadership. Whatever the business or technology domain might be, I encourage the reader to keep an eye out for the best diagnosticians. I think you will find that the best among them could be your own raw materials for the next wave of leaders within your organization.

Best,

Charlie

Decommoditizing Your Technology Career

I was the CTO of a software subsidiary of a large investment bank during the stock market crash of 2007-2008. Before that time, the company had been using a small number of outsourced and offshore technical resources to augment the software development staff. The parent company had established an offshore technology center in Hangzhou, China and I had done preliminary reconnaissance for obtaining staff from there. We also used resources from a second tier Indian outsourcing company to support software testing. Up until the market crash, our use of outsourced and offshore resources was targeted and fairly modest. The market crash changed all that.

Market forces changed the focus to cost cutting which included staff reductions at all levels. A mandate from on-high regarding new staff requests followed the reductions. The new staffing request policy required an explicit justification for the hiring of onshore staff. The message was clear: If we wanted to hire new technical staff, getting them via offshore or outsourced means was the path of least resistance. The theory was that offshore/outsourced staff was 30-50% as expensive as similar onshore staff and that costs had to be reduced. Management viewed some technical roles as fungible commodities that could be easily replaced with cheaper staff in lower-cost locales. Debating the wisdom and economics of this argument is beyond the scope of this article. Suffice it to say that I was a good lieutenant and knew how to pick my battles. After five years time, over 70% of the software development shop was split between China and India.

Throughout that period, I had to ascertain which roles needed to be kept in-house and which roles were commodities that could be externally sourced. Employees continually expressed concerned that their jobs might be moved offshore and were looking for career advice on how to avoid being displaced.  The answer was fairly simple. We could not risk offshoring/outsourcing key roles that added unique value to the business, and I frequently advised the staff on this point. Those roles fell into a few buckets as follows:

Customer-Facing

We had a small group of technologists who did exceedingly well at facing off with customers. They could both evangelize our technical solutions to customers and communicate customer technology needs back to the business. In my experience, this is a rare skill for a technologist and is a key differentiator in supporting the sales organization. Further, the industry had changed over the preceding ten years to a point where the technologists representing the customer had an almost equal say at the negotiating table to those representing that customer’s business needs. Having our technologists facing off with the customer had become critical to the success of the deal.

Solution and Product Architecture

As a financial software company, our products and solutions were the “secret sauce” to ensuring future business success. Offshoring/outsourcing any role supporting the formulation and evangelism of either solution (products + services) or product (technical) architecture represented a risk to the business. People in these roles required a deep understanding of the business domain, knowledge of the capabilities of the company, and the skills to synthesize that knowledge and understanding into value-added solutions and product features. These skills take a long time to cultivate, and we needed to keep these roles close to home if the business was to thrive.

Project Management

Managing complex projects is a skill requiring attention to detail, a focus on the goals, and the ability to collaborate with a diverse set of personalities. Globally distributed resources created by the increased use of offshore/outsourced staff added a new layer of complexity to this job. We prized people who could manage projects in this environment. The best of these folks understood our corporate strategy and goals. They also understood the dynamics of the organization knowing where the pools of expertise existed and how to maneuver inevitable political and cultural obstacles. In short, they knew how to exert organizational influence to align project resources with execution to achieve corporate objectives. It was imperative that these people remained in-house and that we retained them.

In Hindsight

The years surrounding the stock market crash of 2007-2008 were challenging. We made a lot of tough decisions that displaced good people. The times forced us to take a cold, hard look at the many roles that made up a successful software development business and make choices. In the end, the remaining technology staff positioned themselves and the business for success by gaining a new understanding of what constituted value to the business. While the lessons were hard, I think many technologists emerged with a new appreciation of their role within the business and what was needed to achieve superior results while avoiding becoming commodities.

Best,

Charlie

Considering Your Technology Career. Part 1 of 4: Are You Satisfied?

As of this writing, the job market for technical talent is booming in the United States. Even though the US economy is still recovering from the economic calamity of 2008, circumstances have conspired to create a high demand for technical talent. The fear of outsourcing has driven down enrollments in domestic technology education while the pace of technology change continues to grow within US businesses. Those who embarked on a technology career during these times are now seeing demand for domestic talent while the supply of such technologists slowly catches up. As with all boom and bust cycles, this will change over time. But for now, it’s largely a sellers’ market when it comes to career mobility.

During these high-demand cycles, the technologist’s fancy turns to job satisfaction and career mobility. As my colleague Theresa Hummel-Krallinger writes, companies realize that they must compete for tech talent and are starting to wake up and smell the improving job market. In the meantime, ambitious employees assess their current positions and their prospects.  The purpose of this article is to share a model that I devised for assessing job satisfaction that helped guide me in my career decision-making.

It is my sincere hope that the reader will earnestly look at their level of satisfaction with their career. My experience suggests to me that satisfied employees are energized by their work and produce better results. Those are the type of employees that a technology leader should aspire to attract and to strive to retain. For my colleagues who might question the wisdom in suggesting that technologists assess their satisfaction with their careers for fear that these employees might be tempted to depart, I say that employees are already thinking about their positions when circumstances create dissatisfaction and recruiters knock on their door. My sincere hope is to guide technologists to make more informed career choices.

A Model for Assessing Satisfaction

Over my many years of employment in the technology sector, I’ve changed jobs a number of times. Each change was preceded by events leading to a sense of dissatisfaction. This should come as no surprise to anyone who has changed jobs through their own volition. After each dissatisfaction-provoking event, I did some soul-searching about my current situation and whether it was worth staying or leaving. In most cases, I came to the conclusion that things were not as bad as I first thought and ended up staying. In a few instances faithfully documented on my resume, I pulled up stakes and moved on. When I did ponder my situation, I considered it along the four dimensions of interest in the work, how I was compensated, the level of politics and bureaucracy present, and the prospects the job held for the future. The following illustrates those dimensions:

satisfaction

After getting past the emotional component of my apparent dissatisfaction, I thought about each of the four dimensions to try to assemble an objective view of my situation. I’m sure that many who read this article will come up with additional dimensions of their own. But these four kept resonating with me and provided a sound model for my decision-making.

Interest

Job interest is a subjective topic. For me, jobs that provided the opportunity for technical innovation are important. I’ve always been a bit of a technology wonk, and I gravitate toward the interfaces between software, infrastructure, and humans. This had lead me to interest in user experience (UX) and what is now called development operations (DevOps). I’ve also spent time early in my career building tools to support other developers. This highlights my interest in seeing the things that are developed applied to real world problems.

My interests aren’t necessarily your interests. I challenge you consider what you really find interesting in your career and your current situation. It might be the excitement of technical innovation, but it could very well be the chance to build a cohesive team, the prospects of learning from really gifted architects, working within an engaging business domain, or any number of things encounter through the course of technology development. Only you know what interests you!

Compensation

Compensation can be a dicey topic. Many people convince themselves that sub-par compensation is the main source for their dissatisfaction. In practice and having known scores of very talented technologists, I have found that compensation is rarely the primary cause for dissatisfaction. More often, dissatisfaction with other dimensions is amplified by thoughts of better compensation. I caution the reader about this because I have seen employees job-hop for pay and get themselves into worse situations because they didn’t understand the true source of their dissatisfaction. Of course, that’s what this model is all about!

When considering compensation, it is valuable to understand the market value for your role. There are many sources of salary information available that is beyond the scope of this article. Suffice it to say that it’s not too hard to understand if your compensation is equitable. However, salary is only one component of compensation. In today’s economy, it’s wise to think in terms of “total compensation.” Total compensation is a fairly recent approach where you consider all sources of income and benefits offered by your employer as a single package. This includes salary, incentives, paid time off, retirement contributions, health and other forms of insurance, and any other instance where you get a financial benefit as part of the terms of your employment. This is especially important now in an era when employers expect quite a bit from employees, and healthcare costs are rising. Not all employers are equal, and the benefits add up for both individuals and families. Viewing remuneration through the lens of total compensation allows you to balance all facets of a package. For example, a lower base salary might very well be offset by a better incentive compensation program and vice versa. You might have a young family so work-life balance in the form of paid time off, and better healthcare may be paramount. It depends upon your individual preferences and circumstances.

Bureaucracy

We all have parts of the job that we like and dislike. I have noticed that for me and many technologists, few things are more annoying than excessive bureaucracy and the related politics. Silly things like jumping through hoops to get office supplies, decent equipment, and competent support can irritate an employment experience. Admittedly, I have rarely found bureaucracy to be a primary cause for dissatisfaction. But taken holistically, it can sometimes tip the balance when considering an employment situation. Conversely, we frequently take for granted well-run bureaucracies that provide great service. I urge you to take a balanced approach when considering this dimension since every company has its challenges in this area. The bureaucratic grass is seldom greener elsewhere; it’s just different grass!

As a final note and a pet peeve of mine, please treat your administrative and support staff well. All too often these are the folks who contribute the most to the efficient operation of the business. They seldom get the credit they deserve when the business runs smoothly but sure take it on the chin when things don’t go as expected!

Prospects

Of all the four dimensions, assessing that prospects of your role within the company is the most subjective of all. As I previously mentioned, being involved with innovation is a great source of job satisfaction for me. I tie my prospects for continued employment to the company’s commitment to sustainable innovation. You need to ask yourself what matters to you. If it’s not innovation, then is it long-term stability, continuing to work with a great team, advancement up the career ladder, predictable compensation increases, short commute, amount of business travel, having a great manager, or some combination of these! Whatever it might be, remember that change is at the heart of most businesses. Today could very well end up being the “good old days”, or the future could hold the promise of better things to come. Only you can judge.

For Example

I had worked for a software company for many years. I joined right after they received a round of venture capital investment. I stayed through their acquisition by large firm and eventually became a technology leader within the division. The parent company left us alone so long as we generated revenue and margins, so the prospects for the future were reasonably bright. We had the stability of a large firm yet continued to operate as a much smaller company, and I had a great relationship with my manager. My satisfaction profile looked like this:

before1.png

We had just completed a major product uplift, and the work continued to be interesting within a range of variability from average to high (see the above length of the “Interest” indicator line). I felt adequately compensated for the position when viewed through the lens of total compensation. The raises were low, but the incentive compensation was very good so long as we performed. The bureaucracy was well-run and supportive because the team had been together for a long time and was very cohesive. The prospects were fairly bright within a range of uncertainty. In short, the averages seemed to be in my favor.

Then two things changed. First, we were placed under a new senior manager at the parent company. Second, we were looking for a modest investment from the parent company to continue to innovate the product. To make a long story short, it turned out there would be no investment from the parent to support the proposed innovation strategy and it was clear that the parent company would be calling the shots more and more. They would be taking the division in a different direction. After assessing my satisfaction once again, the profile had changed:

after1

Since innovation would be an uphill battle, my interest level dropped. I knew that we would have new direction from above so the bureaucracy and politics would rise as is common with larger companies. Finally, I saw my prospects for future advance diminish as I would be a smaller cog within a bigger wheel. I decided that it was time to seek greener pastures.

What’s Next?

I hope you will take the model described above and apply it you your situation. You just may find that you have a pretty good deal where you are now. I found that to be true most of the times I found myself soul-searching about a job change. Remember that petty annoyances at work are mostly ephemeral and evaporate after a few good nights sleep. However, if you keep coming back again and again to the notion that it might “be time” then I hope you will use the model to create your own satisfaction profile. It might guide you to a better opportunity!

In the three articles to follow, we will examine three main stages in the lifecycle of a company: The Start-up stage, the Take-off stage, and the Maturity stage.

company1

We will use the model for assessing satisfaction introduced in this article to assess possible sources of satisfaction and dissatisfaction within each of the three company lifecycle stages. My intention is to help you understand the different types of companies and to be better able to assess what might be best for you. You could very well find that your present situation is just where you want to be, or you could find that your interests lie elsewhere!

Stay tuned!

Best,

Charlie

Better, Faster, Cheaper: Picking Three

The old chestnut in software development goes, “Do you want the software better, faster, or cheaper? Pick two!” This dilemma has increasingly plagued technology executives as the pace of technology change accelerates, and the business challenges the software delivery organization to “pick three.” Historically, software development organizations have balanced these apparently competing dimensions by assuming that such balance was a zero sum game where one dimension must be traded off for the other two. In this article, I will challenge the conventional wisdom and discuss a risk-based software development management model where we assess the risk to on-time delivery and mitigate those risks to keep the project on track. In doing so, we more efficiently use software development capacity ensuring that the optimal time is allocated to each software development activity to ensure that we deliver high-quality software. I call this the “Delivery Risk” model.

The Relationship Between Better, Faster, and Cheaper

As a technology executive responsible for software delivery, there are three primary responsibilities I have to the business that I serve. First, and because I work in the clinical trial software domain where patient safety and data integrity are paramount, I have the responsibility to deliver high-quality and defect-free software. Next, since our software is used to generate top-line revenue, I have a responsibility to the business to deliver our software within predictable timelines so that our production, support, and business development organizations can operationalize the software and plan their promotional and sales activities. Finally, since the software development organization is a cost center, I have the responsibility to exercise disciplined stewardship over the resources entrusted to me so that I don’t adversely affect margins beyond the costs predicted in the budget. In summary, the business holds me accountable to balance “better, faster, cheaper” and to strive to deliver on all three.

The primary reason for the “pick two” nature of the relationship between “better, faster, and cheaper” is because much software is delivered late and can be of questionable quality. When this happens, software development capacity is lost to subsequent late project starts and to capacity that must be spent fixing defects when it could be used developing valuable new features. These two impacts conspire to raise the cost of software development. Further, as a software development project becomes late; the slippage is frequently mitigated by limiting activities that that tend to occur late in the software development cycle like integration testing. Pinching quality-enhancing activities reduce the overall quality of the software. In short, the key to delivering software more cheaply is to keep projects continuously on track to ensure that all software development activities, especially those that improve quality, are not subject to short cuts.

In the Delivery Risk model, we seek to identify and monitor the events that conspire to make software projects late and mitigate their effects early enough so that they don’t adversely affect the estimated delivery date. The events to monitor are requirements scope, resource availability impacts, and delivery risks. There are three assumptions that serve as precursors to implementing the Delivery Risk model. First, we assume that the software teams can provide reliable software estimates. We provide time to work out the design before the team commits to a date. Second, we assume that an aggressive software quality assurance mechanism is in place. In our case, we use aggressive automated unit testing covering more than 95% of the code to ensure that defects introduced by new code are detected and corrected quickly. Third, we assume that the product managers appropriately manage project priorities. Once a project is underway, it must not be preempted to service higher priority development efforts. Project “context-switching” not only drains development capacity but it also adversely affects staff morale. Estimation, automated testing, and project prioritization merit discussions of their own and are beyond the scope of this article.

The Delivery Risk Model

The intent of the Delivery risk model is threefold. First, it identifies tactical impediments to software delivery that need attention. Second, it identifies trending potential impediments to software delivery that might need attention. Third, it identifies and classifies risks that need monitoring and mitigation. The overarching goal of the model is to identify and track risks while there is still time to react and before their effects become detrimental to the overall delivery of the project. In short, the Delivery Risk model seeks to give early warning if the train is running off the tracks!

The first component is the tactical assessment of risk across all projects slated for the proposed release. In this assessment, we track scope, capacity, and delivery status.

tactical

In the above sample, each project contains a RAG (red/amber/green) status for each status element as well as the estimated delivery date. Scope status refers to the requirements or use cases that make up the project. Failure to lock down scope is among the highest risks to timely completion. In cases where an external customer provides requirements, as is common with data integration projects, we often track scope more granularly to ensure that the customer has agreed to the requirements and the development teams thoroughly understand them. We are less granular when the subject matter expertise is internal to the company. Resource status refers to the project staff itself. Resource risks can arise from unexpected staff attrition, illness, or paid time off. The delivery status refers to the project burn down as we will see shortly.

The five sample projects listed above are managed as a proposed release that we often refer to as the “program.”. The program team meets weekly and uses the representation above to discuss the current status of the project.

The second component is an assessment of the overall trending of each project comprising the release. For each project, we are especially interested in the deviations from expected burn down of project use cases against estimates. In a typical burn down chart, we compare ideal to actual burn down within an iteration:

Burn_down_chart

In our case, we are interested in the deviations from expected burn down of project use cases against expectations. In short, we are more interested in the deviations around the ideal burn down rather than the burn down itself using the ideal burn down as the zero point on the x-axis and comparing against actual burn down on the y-axis. Further, we want to track and watermark the deviations across iterations to get a holistic view of how we are tracking against the target delivery date:

delivery

In the above example, each iteration is listed on the x-axis. The y-axis represents the deviation in use case burn down (velocity) from the expected team velocity needed to achieve the estimated delivery date. We translate this into a percentage increase or decrease from the expected delivery date. We then watermark the percentages to establish the RAG status. Finally, and to get a clearer view of the source of the risk, we break out the data by development role including Design, Development, Testing, and Product Management. We can do this because our SDLC contains process gates that allow us to track use case transitions through the process. There are some artifacts to note above. For example, if an activity has not started, it shows up as pegged to 100% overrun as in the case of Product Management approval of completed use cases. We know that such approvals are trailing indicators, and we are not typically alarmed by them unless they persist for long periods of time.

The delivery risk chart is probably the single most important tool in the model. It enables the team to see events trending over time. We have some simple rules that we apply to the trends. Green means that no action is needed. Amber means that we need to watch and plan for mitigation. Red means that we need to act on our mitigation plans. Using this tool helps us avoid a “happy path” mentality by showing us how we are trending well in advance of the delivery date. The team is constantly evaluating risks to delivery and considering mitigation strategies and helps us to avoid “fire drills” at the end of project since it spreads mitigation throughout the project.

The final component is the enumeration and assessment of potential risks. In our case, we maintain a risk register and review it weekly. When properly managed, the risk register contains those risks that the team needs to overcome to keep the project on track, rather than risks that caused the project to be late. Here is an example:

risk

In our example above, we classify each risk for its probability of occurrence and the impact should it occur. We then note the strategy for dealing with the risk should it become real. Again, these are reviewed and adjusted weekly.

Cautionary Notes

I need to caution the reader on the use of the Delivery Risk model since there are limits to its use.

The Delivery Risk model IS:

  • A means to visualize current delivery status and risks.
  • A means to visualize trending status to allow staff to react earlier.
  • A means to ensure that all development activities get appropriate capacity attention, especially testing.
  • A vehicle to identify process improvements as part of a holistic, continuous improvement discipline.

The Delivery Risk model IS NOT:

  • An exact science. It is the combination of art and science intended to identify risks and mitigation early. The model is deliberately organic and depends upon the combination of data, insight, and trust.
  • A team or project performance metric. It should never be used as part of formal goal planning for individuals or teams. Nor should it be used as a mechanism to punish or intimidate teams or individuals.
  • A magic bullet. It can help you limit project overruns but may not help you avoid them altogether.

Credits

Bob Ponticiello and Phil Gower first proposed the Delivery Risk model while working at Princeton Financial Systems. Bob and Phil managed a large globally distributed software development effort and noticed that software quality seemed lower the later the software was delivered. They took years of waterfall delivery data and created a risk factor spanning many years of software releases. The “risk factor” was expressed in days expended beyond expected waterfall milestones (requirements, design, code, test) and was cumulative over the life of a project. They graphed the risk factor against the post-release defect count:

correlation

Their data revealed a 0.6 correlation (see the trend line) between risk factor (lateness) and post-release defect count (quality). We tested whether the relationship was causal by deliberately managing delivery risk against key waterfall milestones in subsequent projects. The theory was that if we managed to waterfall milestones, we would ensure sufficient time was allocated to each software development activity, especially thorough software testing. After a very short time, we were able to demonstrate that managing the risk factor not only brought the software in on time but allowed us to deliver higher quality software. I borrowed Bob and Phil’s idea and adapted it to Agile development methods. I feel fortunate to have worked with two such insightful and committed software development leaders.

Final Thoughts

The Delivery Risk model is an aid to getting projects completed on time while mitigating the risks to quality. When it comes together, it helps makes the most efficient use of a software development organization that translates directly to cost efficiency. Further, when used as an aid to continuous improvement, it can tap into the teams’ common commitment to delivering great software while promoting team communication and building trust. I truly believe that when open and honest communication takes place within a software development organization, the business can get results better, faster, and cheaper and will never have to settle for just picking two!

Best,

Charlie

Designing an Accountable Software Development Organization Part 3: Roles, Organization, and Culture

In part 2 of “Designing an Accountable Software Development Organization,” I discussed creating a vision for the software development organization that serves as the desired end-state. I explained that levers and tensions exist between the organizational components of the end-state model as well as how they could be harnessed to drive stakeholder behavior in healthy directions. In this third and final article. I will show how to align the roles and organization of a software development organization with the levers previously discussed. Finally, I will enumerate a set of behavioral guideline that help to keep healthy tensions healthy while avoiding unhealthy tensions.

Roles and Organization

As a software engineer by training and education, I have always believed in a pragmatic “form follows function” approach. With few exceptions, I’ve been fairly successful when applying it to organizational dynamics or software development organizations. My goal then is to define roles and align organizations along the natural lines of the levers I previously defined. This way, I assign authority and responsibility that is inherent within the vertical domains directly to directors and their organizations. This alignment encapsulates authority and responsibilities within the discrete organization and assigns leadership to a single director who manages his/her team, negotiates healthy tension-related issues with his/her peers, and provides me with a single point of contact to report on the status of the lever under their care.

In my software development organization, I have defined three key organizations: product management, technical delivery, and operations. The roles flow from the responsibilities and authorities defined earlier:

roles

I have made a few accommodations to a straight one-to-one alignment based upon the mission and skill sets. First, because my mission is to create a next generation product while maintaining and supporting the existing product, I have created two technical delivery teams. I found it prudent to ring-fence development staff within product domains to avoid having staff distracted by issues outside of their respective domain. For example, when I have had product teams intermingled, there was a temptation to divert key next generation product staff to solve knotty problems within current product domain. All too often, this proved a distraction to the mission of developing the newer product that I sought to avoid. Second, since the roles of risk tracking and process adherence are closely related without having inherent tensions between them, I collapsed both roles into an “Operations” domain and leverage the project managers in both roles.

Culture and Ethos

Now that we have defined the roles and organization along the lines of the levers, it’s time to turn to the management of tensions. The essence of developing great software lies with balancing the various forces at work within and between the organizational components. I have designed my organization such that healthy tensions bubble up to the director-level within each organizational role while minimizing the occurrence of overlapping unhealthy tensions. However, where there is tension, there is the opportunity for conflict. In keeping with my goals of engendering communication and trust, I have a simple set of rules for dealing with this that focuses on rules of engagement as well as encouraging professional maturity and development.

As described in the levers and tensions section, I make the responsibilities and authorities of each role explicit. I take a “carrot and stick” approach to describing the behaviors I expect and those that I wish to avoid. I set clear expectations for the directors as in the following example:

ethos1

Setting clear expectations about authority and responsibility boundaries is frequently reinforced within the organization. The directors understand this and set development goals for their staff accordingly.

When inevitable conflicts do occur, I prefer to set expectations for appropriate means of resolving them. At the risk of sounding “preachy”, our business partners and peers expect and deserve to be engaged in a respectful and dignified manner even when disagreements arise. If a software development organization is to communicate effectively and engender trust, I believe that a high level of personal integrity must be maintained in interpersonal relationships. If we don’t behave appropriately, then we have little right to expect trust and respect in return. We have all seen recent reports where a major automobile manufacturer was caught cheating on emissions tests after marketing their engines as clean. This was a clear example of where a technical delivery organization broke the trust with their consumers by acting with low integrity. Consumers are correct to no longer trust their claims.

I provide a simple set of guidelines for engaging staff, peers, and their managers. I again mention behaviors I expect that those I wish to avoid:

ethos2

I think most of these speak for themselves. I would like to comment on “taking the high road” a bit further. There are political issues and rivalries in any organization. Frequently, it is easy to take the approach of “fighting fire with fire.” I reject this approach for the simple reason that while such an approach might be emotionally satisfying, it will almost inevitably lead to regrets. Regret, whether in personal or professional affairs can be emotionally corrosive and ultimately self-defeating. I try to avoid situations leading to regret whenever possible.

Getting Results

So what results can you expect from a truly accountable software development organization? In my most recent experience, the software development teams saw positive quantitative and qualitative results over a three-year period. The teams’ on-time delivery went from hitting the mark less than 60% of the time to achieving on-time delivery over 90% of the time. The software quality as measured by post-delivery fixes increased by over 75%. While we applied the Agile Scrum methodology to an inherently risk-averse business domain, we have consistently passed client audits with only minor findings.

Qualitatively, our business partners report that they are more consistently satisfied with the software solution. This increased satisfaction has led to increased confidence in the software development organization. The software organization itself has developed a new confidence and sense of satisfaction knowing that they deliver high-quality solutions when the business needs them.

My experience is limited to software development and infrastructure technologies and I have applied the above techniques within those domains. However, the principles of empowerment and alignment should apply to any business domain. I would love to hear back from those who try their hand at the application of these principles!

Best, Charlie

Designing an Accountable Software Development Organization Part 2: Vision, Levers, and Tensions

In part 1 of “Designing an Accountable Software Development Organization,” I laid the foundation for software development accountability by translating the business needs required from the software development organization into a clear statement of purpose. I went on to define tenets and principles to support the mission and underpin its implementation. In this article, I will discuss creating a vision for the software development organization that serves as the desired end-state. Levers and tensions that exist between the organizational components of the end-state model will be covered as well as how they can be harnessed to drive stakeholder behavior in healthy directions. Finally, since behaviors can become unhealthy, I will discuss the source of such behaviors.

Vision

Other than in cases of natural staff attrition, new software development leadership is installed when the software development organization has not met the goals of the business that it is entrusted to serve. The new leadership has an opportunity to reset the organization to meet those needs. In almost all cases and after some reconnaissance, the new leadership defines a new organizational end-state. That end-state is intended accomplish specific goals. First, the end-state serves to operationalize the mission of the software development organization. Second, it points to realistic expectations and outcomes for the organization. Third, it provides a baseline for a regimen of continuous improvement. Finally, it helps establish a clear line of sight to the business goals and helps each member of software development organization understand how they contribute to those goals. In keeping with the principle of subsidiarity I discussed in the previous article, I recommend that you share the development of the end-state with your software development organization as it’s developed. Doing so both enlists the staff’s support in realizing the vision and gives them a chance to add criticism and correction in cases where you might fail to understand nuances in the existing culture or personalities.

In the example I used in the part 1, I was hired to shape a historically consulting-based software development organization into a product-centric and customer-focused one. The primary goal was to create and deliver new high-quality, innovative solutions predictably to the business while maintaining and enhancing existing products. I will add that this was in the domain of software supporting pharmaceutical clinical trials. For those who aren’t familiar this business domain, patient safety is of paramount concern. Because of this, it is among the most highly regulated and audited sectors of the software development space. Consumers of the software tend to be highly conservative and risk-averse. For example, customers routinely audit the software and the SDLC to ensure complete traceability between requirements and tests as well as to ensure that the SDLC is sound and followed meticulously. (As an aside, we implemented an Agile Scrum-based SDLC discussed in the article, ”Agile/Scrum Adoption in Regulated Industries.”) To address the patient safety concerns, we needed to produce exceptionally high-quality software that was well fit to the business need. To address the regulatory needs, we needed a strong focus on the software development lifecycle and its governance. We also needed to continue to support the existing product suite even as we developed its successor. It was clear that I needed strong alignment of authority and responsibility within the requirements, technical delivery, and SDLC governance domains. The organizational end-state was as follows:

vision

The gray boxes to the left represent the sources of business requirements feeding the product pipeline. The arrows at the top represent the product development process including product management and development steps while the boxes below represent the organizational landscape needed to support the process. In short, this depicts the end-state for the software development organization. It also helps underscore the strategic goal of reliably delivering innovation while pointing to the organization needed to realize the tactical implementation.

Levers

The leaders of a software development organization constantly adjust the direction of the organization as the needs of the business change and as risks arise. They do so by exerting leverage on key areas of influence. In the case of the organization I was designing, I identified 4 “levers.” Requirements fidelity refers to the fitness for the business purpose and prioritization of the feature pipeline. Technical delivery refers to the development capacity, timelines, architecture, and quality. Process adherence refers to SDLC governance including audit fitness. Risk tracking refers to the reporting of project status and risks. The responsibility and authority within the domains are as follows:

levers

As you can see, it was crucially important that the authority and responsibility aligned vertically within each domain. This vertical alignment ensures that each domain can act independently within their area of responsibility and drive efficient execution. The domains intersect at areas where healthy tension is required to balance the responsibilities of the domain. Those intersection points are precisely the leverage points where influence is best exerted. You will also notice that the intersection points are also places where the most discord can occur within a software development organization. These intersections were placed purposefully to ensure that healthy tensions rose to the level of the technology leadership visibility rather than being hidden from them.

Tensions

I spoke of healthy tensions in the last section. In my organizational design, I wanted requirements fidelity, technology delivery, risk tracking, and process adherence to act as independently as possible while having the freedom to report risks without fear of consequence. For example, those responsible for the requirements have the goal of ensuring the product meets the needs of the customer. If some requirements complexity made it harder to develop technically, then I wanted that tension to be explicit so it could be adjudicated at the right level and by the right people. By the same token, insufficiently detailed requirements could create risks to timely technical delivery that needed similar adjudication.

I say these are “healthy” tensions because balancing them is necessary for the development of great software. However, I have seen organizations where the authority and responsibility overlap to create unhealthy tensions. Here are a few examples:

tensions

In the above example, I have taken the four levers and shown what can happen when inappropriate superiority/subordination occurs between the domains. For example, I once inherited a technical delivery team where the project manager reported to the technical delivery management. In some cases, projects were delivered late. I discovered that that project manager did not feel free to report risks to the delivery timeline leaving those risks largely hidden. Clearly, the project manager felt that there could be consequences if risks were reported that painted the technical delivery team in a bad light. Leaving these risks unaddressed was frequently the source of late technical delivery. Conversely, I have seen organizations where the project managers had the authority to set delivery timelines on behalf of technical delivery teams solely based upon when the business wanted the software delivered. It was no accident that the technical delivery teams seldom met the timelines since they were never given the opportunity to produce realistic estimates. In both of these examples, the morale of the staff suffered because they felt, correctly I might add, that they lacked the authority to execute on their responsibility to the business. The project manager in the first example and the technical delivery team in the second felt victimized. The remainder of the unhealthy tensions speak for themselves.

When designing your organization, it is important that you align the vision for the organizational end-state with the mission of the software development organization. The organizational components should balance vertical authority and responsibility with the requirement to make healthy tensions explicit. Care should be taken to avoid authority and responsibility overlaps that create unhealthy tensions that could sabotage meeting responsibilities and produce low morale. It can be a delicate balancing act. But when done with a deft hand, it can ensure that execution is efficient, and morale remains high since the teams have the proper authority to ensure that they can meet their responsibilities. Accountability is the natural outcome of a successful balance of authority, responsibility, and adjudication of healthy tensions. As each domain delivers on their responsibility, their confidence in their delivery capabilities grows. High morale for the software development organization and trust from their business partners are natural side effects of true accountability.

Preview of Part 3

In part 3 of this series, I will discuss aligning roles and responsibilities along the lines of the organizational levers. I will also describe engagement guidelines that can create a culture or “ethos” intended to keep healthy tensions healthy while limiting the expression of unhealthy tensions.

Stay tuned!

Designing an Accountable Software Development Organization Part 1: Tenets, Mission, and Principles

All too often the word “accountability” is associated with the word “blame” when used in the context of software development organizations. In these situations, the software development organization was viewed as somehow failing to meet the business need. Either the software solution came in late, or it contained insufficient features or quality to pass muster. Sometimes the software was seen to fall short on each dimension. When examining the causes of these failures, I have found that they usually result from three broad categories of issues. The mission of the organization was unclear, the technology leadership did not align the software development organization for success, or those same leaders did not empower their staff to be successful. Frequently, all three were the case.

In my previous article, “The Executive and the Software Craftsman,” I discussed how technology leaders can prepare their software craftsmen to make better business decisions. In my subsequent article, “The Technologist’s Guide to The C-Suite,” I went on to explain what the business expects from software craftsmen and other IT professionals. In both cases, it was important that the business and technology leadership built and maintained a strong relationship with their technology delivery staff based upon communication and trust. In a series of three articles, I will lay out a framework for harnessing that trust into an accountable software development organization by accomplishing three key goals:

  • Clearly enumerating a mission for the software development organization to ensure that all parts of the organization are pulling in the same direction. Basic tenets and principles must be spelled out that support the mission and underpin implementation.
  • Creating a coherent vision for the software development organization that defines the desired end-state. Flowing from the vision are levers and accompanying tensions that must be placed in proper alignment and managed to reach the desired end-state.
  • Aligning roles, culture, and ethos to help keep the tensions healthy while continuing to drive the organization in the right direction.

Tenets

As with many other technology executives, I have had the benefit of working with and for some talented managers. Along the way, I have synthesized some basic tenets that guide my organizational design and development. These include:

  • In a principle called “Subsidiarity” problems are best dealt with at the most immediate or local level consistent with their solution. L. David Marquet describes this as locating authority close to the information needed to make decisions in his talk on Greatness. In a software development organization, this requires that the staff is armed with the best information to make good business decisions. Establishing line-of-sight to the business problems to be solved is critical to arming people with the best information to make decisions.
  • Healthy tension exists between departments and when properly aligned, can provide checks and balances to yield better outcomes. The tensions must be identified, explicitly called out, and managed, so they don’t become unhealthy. The assumption is that tension is a healthy and necessary part of the software development process. Misalignment or mismanagement of healthy tensions can quickly turn healthy tensions into unhealthy ones.
  • Clarity of purpose within the organization, alignment of responsibility and authority with a departmental domain, and management and adjudication of healthy tensions between departmental domains are the antecedents of accountability.
  • Good and bad culture (Ethos) is no accident and is a consequence of setting expectations for and management of healthy and unhealthy tension, or a failure to do so.

Mission

I’m sure most readers helped develop departmental or corporate mission statements and had rolled their eyes along the way. I’m no exception. All too often such statements are developed by committee and end up being watered down or so esoteric as to be meaningless. However, when done right, such statements can be essential to providing valuable context and clarity needed to align a software development organization with the business domain it serves. I have developed some simple questions and rules for developing mission statements for software development organizations that I have found helpful:

  • How does your software development organization specifically contribute to the business?
  • The mission should inform both organizational strategy and tactics.
  • Five or fewer bullet points should be sufficient to capture the essence of the mission.
  • The mission statement should be a reflection of the technology executives character and not reflect group-think.
  • The mission should stand the test of time and not significantly change from year to year.
  • The mission should be reinforced repetitively with the staff at every opportunity.

As a concrete example, I was hired at my most recent job to shape a historically consulting-based software development organization into a product-centric and customer-focused one. The primary goal was to create and deliver high-quality, innovative solutions predictably to the business. After two months of reconnaissance, I developed the following:

mission

This mission has not changed in close to three years and continues to provide clarity to the department. I also reiterate it at every opportunity, especially during departmental briefings and strategy/goal planning sessions.

Principles

Just as I have development tenets that guide and inform my organizational design and development, I have developed principles that guide the development of great software. I use these to help my staff understand “what makes me tick” and help them understand what I expect from them and how I make decisions. As such, I humorously call them “Charlie-isms.”I like to think that some of these principles have helped in the career development of my staff. These are:

principles

I will not delve deeply into all of these, but two are particularly near and dear to me. Since I was a developer myself for many years, I have a running joke that goes, “Software developers have two jobs: Fixing bugs and creating bugs.” When we consider this seriously, unfit solutions and low-quality code sap the development capacity of a software development organization. A persuasive business case for solution and code quality beyond the obvious ramifications emerge. Namely, that the production of quality software means that you can devote more development capacity to developing more great software rather than fixing what you’ve previously built. Hence, “Product quality increases development capacity.” There are sometimes valid reasons for taking quality shortcuts especially when timelines are critical. However, delivering low-quality software is seldom beneficial in the long-term.

As a second point, development organizations frequently feel besieged by their business partners when the software development organization does not meet the goals of the business. When this happens, the confidence of the software development organization can erode and lead to a culture of negativity within the organization that manifests itself as “no, but” responses to requests from the business. For example, when a software development organization is asked if they can deliver a given software component within a fixed time, the response will frequently start with a “no.” “No” is then followed by the enumeration of the conditions that will lead to a successful outcome. By the time the discussion turns to meeting the request, the people making the request have already decided that the software organization is an obstacle to progress rather than a partner to success. I counsel software development organizations to understand that most requests are reasonable given the right combination of requirements, time, and staff to deliver the solution. When engaging the business, I turn the conversation into one where the software development organization discusses costs of delivering the solution to meet the goals of the business. Once the business understands the costs of meeting the request, a “yes we can” rather than “no we can’t” tone becomes apparent. The business then views the software development organization as a partner committed to understanding and meeting common goals.

Preview of Part 2

In the next article, I will discuss creating a vision for the software development organization that serves as the desired end-state. Levers and tensions that exist between the organizational components of the end-state model will be covered as well as how they can be harnessed to drive stakeholder behavior in healthy directions. Finally, since behaviors can become unhealthy, I will discuss what can be done to prevent that from happening.

Stay tuned!