Tuesday, September 2, 2014

What’s the difference between Verification and Validation, and how do I figure out which project practices support which process area?

[Dear Readers, for the past several months, our good friend Pat O’Toole, CMMI expert and seasoned consultant, has been collaborating with us on a monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month Pat clarifies some of the misconceptions about Verification and Validation.  Take it away, Pat! ~ the CMMI Appraiser]


[PAT: Note – there are a number of verification/validation misconceptions I am going to address in this article.  In other words, I’m not going to constrain myself to answering the question – but why shouldn’t THAT surprise you!] 

The CMMI addresses this question very explicitly, and yet the question keeps coming up.  In the “Introductory Notes” of BOTH Verification and Validation, the model states: “verification ensures that ‘you built it right;’ whereas validation ensures that ‘you built the right thing.”  All you have to do is follow that advice and you can’t go wrong differentiating between the two.


Yeah, right!  To me, this pithy explanation is “cute,” but operationally useless.  For example, if I’m conducting system testing, am I’m trying to ensure that the developers built it right, or am I’m trying to ensure that they built the right thing.  I THINK it’s both – isn’t it?

Over the years I have developed what I believe is a much more useful heuristic – one that’s 90% “good enough” and certainly much easier to apply.  Consider this alternative explanation…

When performing something that smells like V&V, take a look at who is involved in the activity.  If it’s just our engineers and testers, then it’s probably a verification activity.  The best the technical folks can hope to do is to compare the product they’re building or testing to the requirements everyone agreed to, and look for deviations that need to be addressed.  Verification is ensuring the work products meet the requirements.

On the other hand, if the customer, user, or a customer/user surrogate (e.g., Product Management) is involved in a product evaluation activity, this tilts the scale much more heavily toward validation.  When a customer looks at a system we are building on their behalf, they are much less interested in comparing it to the documented requirements, and much more concerned about whether the product is going to solve their problem, be usable by their people, and run on their already-overloaded computers.  These concerns reflect those of validation – fitness for use and the ability to work in the intended operational environment.

So let’s test this to see if it’s easy to apply and helps us to reach the right conclusions.  If we employed only “Stepford” engineers and testers, we would start each project with perfect requirements that were perfectly understood, build the product accordingly and then perform perfect verification that discovered and remediated all deviations from the requirements.  In such a perfect world there would be nothing left to find in validation!

However in our non-Stepford, imperfect world, I contend that any problems found in validation likely results from one of three conditions:
  1. We missed a requirement; (“You didn’t tell us the pinball machine is for a cruise ship!”)
  2. We misunderstood a requirement (“You meant NAUTICAL miles?”) and/or
  3. Our verification activities failed to catch it (“Oooops!”).
Given this new rule of thumb, which product-related evaluation activities are verification, and which are validation?  See if your answers agree with mine…
  1. Unit testing – it’s just us engineers, which would lead us to verification.  However, in the CMMI, all discussions related to unit testing is found in the Technical Solution process area under SP3.1 Implement the Design – so maybe it’s neither.  Ah, but TS SP3.1, subpractice 4, “Perform unit testing of the product component as appropriate,” has a reference to the Verification process area, so our first instinct was right after all!
  2. Integration testing – it’s typically just us engineers and testers, so verification.  However, if the user, customer, or their surrogate participate in some or all of these activities, that also brings a validation component to this activity.  (A single activity CAN be both verification and validation.)
  3. System testing – same as integration testing.
  4. Qualification / customer acceptance testing – because the user, customer, and/or their surrogate are typically involved in such activities, they are primarily validation.
  5. Prototyping – it depends.  If it’s a prototype like sample webpages or mocked up reports or modified process flow diagrams that are intended to be shared with the customer/user to make sure that we’re going down the right path to meet their needs, then it’s validation.  If it’s an engineering prototype – one used to decide which of three alternative technical approaches will best meet the stringent performance requirements (for example), then it’s verification.
One additional note about Customer Acceptance Testing … some people mistakenly believe that if their customer performs unassisted acceptance testing then validation is covered.  What these people appear to forget, however, is that the CMMI is intended to enhance your organization’s capability to develop high quality products, not your customers’!  Your customers’ ability (or lack thereof) to perform thorough acceptance testing says nothing about your organization’s capability (or lack thereof) to perform validation.

Another common misconception about validation is that “Validation = Customer Acceptance Testing.” Validation’s ‘Introductory Notes’ points out, “…validation is performed early (concept/exploration phases) and incrementally throughout the product lifecycle.”  The CMMI legal lawyers will object, saying that the “Introductory Notes” are only “informative” model components – reminding us that only the goals are required!  To address the objection, gently guide them to Requirements Development SG3, “Analyze and Validate the Requirements” – objection overruled!

I firmly believe that most organizations perform more validation than they give themselves credit for.  To lift the veil on these hidden validation activities, ask questions about customer/user/Product Management involvement throughout the life cycle.  What kinds of things do you get them involved in, and what does that involvement entail?  Do you show them mocked-up webpages, sample reports, etc. to elicit their opinions? 

And what kinds of changes do you make based their feedback?  Questions like these will reveal that hidden validation activities that most development groups perform.

Keep in mind that the CMMI suggests that blissful ignorance of performing such activities is better than nothing – in fact, it’s called “capability level 1” – but it’s not as good as actually doing such validation activities “on purpose” – which is the purview of capability levels 2 and 3.  The model also suggests that relying on project managers or engineers to “do what they feel is right” demonstrates more of an individual capability than an organizational capability.  The CMMI would suggest explicitly establishing and institutionalizing the system of practices that provides an organizational validation capability.  You’re probably exhibiting many of the behavioral patterns already, but the model is encouraging you to support it in a way that ensures it is performed consistently, consistently performed, and improved over time.

I have saved what I consider the most important point for the very end – and that is that the amorphous dividing line between verification and validation is something that is only of interest to us CMMI model geeks and that such discussions should probably not be conducted in front of the engineers and testers.  Not only do we embarrass ourselves when such bickering occurs in public, but they really don’t care!  What they DO care about is developing products that delight the customer.  They don’t get hung up on whether a particular activity is “verification” or “validation” – if it’s the right thing to do, they do it.  And THAT’s as it should be!

© Copyright 2014: Process Assessment, Consulting & Training and Broadsword Solutions

“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton.  Please contact the authors at pact.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed.  New questions are also welcomed!

Monday, September 1, 2014

Would achieving a CMMI Maturity Level help or hurt our competitive difference?

Dear CMMI Appraiser – we are an award-winning federal contractor, providing IT services to NASA, the Department of Transportation (DOT) and Department of Interior. We pride ourselves on being very different from our competitors, but lately our developers have been asking us to consider adopting CMMI for its structured framework. My concern is this: if we were to adopt the CMMI, wouldn’t it hurt our competitive difference and make us same as everyone else with a Maturity Level 2 or Maturity Level 3? ~ Sam C.

Dear Sam,

Great question. The purpose of the CMMI is to help any executive, engineer and/or business professional who is trying to create an environment in which your organization can manage its uniqueness in a structured way. You don’t have to “go for a level” – in fact, unless you need to, don't! The competitive advantages of adopting the CMMI are far more important the mere designation of ML2 or ML3. With a proper adoption of the CMMI, you’ll leave the competition in the dust.



It’s counterintuitive, but true. Rather than making you seem the same as everyone else, you can use the CMMI for guidance on emphasizing and leveraging your difference. The Model helps you establish an environment for allowing you to easily operate like the great company you know you can be – for the long term.

So I wouldn’t worry too much about what other people think, Sam. It’s not about changing your business to suit them. The CMMI is about the transformation of the culture of your company. It’s about improving and changing the way your company behaves, so that you build better products, win new business and retain the customers you have.

For more information on using the CMMI as a competitive differentiator, we are hosting a Webinar that shows you everything you NEED to know about CMMI. Sign up today for: “CMMI - Everything You Need to Know” on Thursday, September 25, 2014 at 1PM EST.

Don't miss this great learning opportunity! Register today.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Sunday, August 31, 2014

SPaMCast Question #4: Does the value of CMMI get lost as a "top-down" directive?

[NOTE: Over the past few weeks, the CMMI Appraiser has been sharing excerpts from a recent conversation with Tom Cagley on SPaMCast about whether agile is resilient – i.e., whether it will be able to spring back into shape after being bound or compressed by the pressures of development and support – and how frameworks like the CMMI can be used to make agile more resilient. Listen to the full interview at SPaMCast 296.] 

Jeff, if you go back far enough, once upon a time, CMMI was being driven bottom-up and was struggling to some extent, the same way we're seeing today with agile.  But when CMMI became a "top-down" initiative, did companies lose sight of its value? ~ Tom Cagley, SPaMCast 

Tom, you and I are both very passionate about CMMI and see CMMI as an important tool for improving processes, and not an end-all be-all tool that solves every single problem. Unfortunately, there are still people who see CMMI as a means to a "CMMI certificate" or to achieve a CMMI rating, which they assume WILL solve every problem.  Rather than receiving the value that the framework is intended to provide – higher quality, faster delivery and predictable, repeatable results  – they treat it more like a stamp of approval, 



So you are correct; the value of CMMI has been lost within some organizations that push its adoption form the top-down.  For proof, just walk around Washington D.C. and talk to any of the six or seven thousand companies that are providing IT services to the government. You’ll find that many of them interpret CMMI as a certificate to be had.

This is primarily a Washington D.C. behavior. Around the country we see some different behaviors, but in Washington the typical approach to something like CMMI – and this also applies to ISO and PMBOK and most of the other process models – is for company leaders to say, “We've got this RFP. We’ve got to get it. We can't do business with the government. We need this certificate.”

When the directive comes down from the C-suite to their managers and staff, it often sounds something like this: "You all shall be CMMI Level 3 by the end of the year!" Or whatever artificial deadline they put on their people. They don’t realize it, but essentially they are saying, "You must be a great company by the end of the year" – which is the true purpose of adopting the CMMI.

But what company can do that? It’s just not logical. So engineers are left to try to figure out how to "pass the CMMI certification" and get their so-called “CMMI certificate.” There's only one place that leads to: A death spiral.

Fortunately, we're not there yet. CMMI is still very strong. It's being used throughout the world in a very productive way in many places. And even in Washington D.C. there are many companies that are enlightened and understand what CMMI is and how to use it as a tool for transforming their culture and changing behaviors to put themselves on the path to being a great company.

But I believe there are still many companies trying to force CMMI from the top-down without understanding the real value of the Model. The same will happen to agile, and the value will be lost, unless we commit to making agile resilient.

For those who would like to know more about making agile resilient, we are hosting a Webinar on September 11, 2014: "Agile Resiliency: Scaling Agile so that it Thrives & Survives".

Like this blog? Forward to your nearest engineering or software exec! 

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Thursday, August 28, 2014

What does CMMI say about Agile cross-functional teams?

Help!!,

CMMI Appraiser, I've been trying to work my google-fu on this topic but cant really get a clean cut answer.

IWhen it comes to CMMI certification and utilizing an Agile flavor development methodology, does CMMI care about what domains are doing what work?

Example)

In Agile / Scrum / XP they preach the concept of cross-functional team memebers. Team members that at any given time can play the role of tester and/or developer. If a team is comprised of all developers (reporting to the app dev family) and this team does not contain members of the test domain (so a QA / Test family) does CMMI have concern with this? would this be considered a conflict of interest?

Thanks, Jesse


Dear Jesse,

Thank you for your question.  It’s a good one!

CMMI is nothing more than a model for improving what you are ALREADY doing!  There is no conflict between CMMI and any development framework or set of techniques like Scrum or XP.  

If you using Scrum, and you want to get more consistent value out of your Sprint Planning and Daily Standups, CMMI has practices to help.  If you’re using Pair Programming or Planning Poker, CMMI can help those also.  It’s method agnostic.  A lot of people struggle with this because much of the “press” about CMMI in the past has been focused around the early large-scale adopters (Aerospace, Federal Government, etc), and they were big “waterfall” shops.  But that was only one of many possible approaches.

There is no rule about cross-functional team members being good or bad in CMMI (in fact, there are no “rules” at all).  There is nothing in the CMMI model that requires independent testers/QA.  

Now, we can discuss whether that is a good idea in all situations, but in a typical small scrum team it usually does not present any issues.

Good luck!


Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead AppraiserCertified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Tuesday, August 26, 2014

Waterfall vs. Agile values – and the Process Thunking Layer

Hey, CMMI Appraiser, when our former boss sold the company, we were a highly effective Scrum team. The new company has a traditional Waterfall approach, and management is telling us they want to see Waterfall behaviors like Gantt charts, big sets of requirements and structured activities. They also say, “But we also want to be agile.” How can they have it both ways? ~ Phil C.

Dear Phil,

The problem you are describing happens when leaders and engineers don’t share the same values. To be useful, values must guide behavior, but when values are out of alignment, so are behaviors. Some executives create the process equivalent of a thunking layer, by which they attempt to have it both ways.


What is a process thunking layer?

When Waterfall-oriented senior executives don’t understand what a Scrum team is doing, they have been known to create a layer of behaviors between the two groups. Similar to a custom written thunking layers like we used to write in the 90s so that 16-bit controls could talk with 32-bit controls in software, the process thunking layer is an interface that adds process overhead and more work to project in order for the two sides to communicate. This layer is not  a part of the functionality, but a necessary evil so they can understand the language.

You are probably seeing this in your organization, Phil. You have a corporate layer with Waterfall values, and a Scrum team with agile values. The thunking layer put in place to translate between the two might take the form of someone whose job it is is to provide that translation (darn, THAT'S why we couldn't have that 5th team member?).

For example, agile teams don’t usually have direct project managers, but if the corporate layer is insisting on traditional project data, they will often hire a project manager between the agile and the Waterfall layers. That’s an example of an organizational thunking layer. They would also realize they need a data layer to translate all the extra data the agile team has to produce because management doesn’t understand the values, methods and techniques of the agile team.

The impact of this type of thunking layer is damaging: Not only is the organization incurring unnecessary process debt and wasting time and money, it is also injecting noise into the system. It creates problems with both project data and requirements. Talk about having it both ways...badly!

So it’s not just lost money and time that cause a problem, but defects in the app that result in lots of rework and animosity between the parties.

Good news! There’s a better way.

To bring your Waterfall-oriented management and Scrum teams into alignment, you can use a values-based architecture that links Values, Methods, and Techniques. The organization will then be able to trace a direct link between the company’s values and how work gets done.

Phil, you can (and should) insist that management aligns the values with the methods and techniques of both Waterfall and agile. One way to do this is by applying a concept we call “Agile Resiliency,” a proven strategy for scaling agile by strengthening and reinforcing and tracing agile values, methods, and techniques. Agile Resiliency is about integrating the architectural strengths of the CMMI with your agile approach to help you make agile resilient enough to resist the pressure to change – and even scale and thrive. The Agile Resiliency Framework removes the thunking layer.

By definition, the Agile Resiliency Framework arms you with the tools you need to help leadership be successful. It provides a landscape for creating positive change by defining the roles of management and Scrum teams and guides the behaviors that everyone needs to exhibit. When you know what the right behaviors are, and what that looks like, you can use the Agile Resiliency Framework to strengthen your agile approach and help your senior execs understand what’s happening and why it is valuable from a business standpoint.

Get more information about helping your executive team develop an Agile Resiliency Framework on our September 11, 2014 Webinar: Agile Resiliency: Scaling Agile so that it Thrives & Survives.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Monday, August 25, 2014

SPaMCast Question #3: Is there a type mismatch organizationally between agile and process improvement methods like CMMI?

[NOTE: Over the past several days, the CMMI Appraiser has been sharing excerpts from a recent conversation with Tom Cagley on SPaMCast about whether agile is resilient – i.e., whether it will be able to spring back into shape after being bound or compressed by the pressures of development and support – and how frameworks like the CMMI can be used to make agile more resilient. Listen to the full interview at SPaMCast 296.] 

Jeff, One of the terms that you used for the imbalance between the way managers versus developers see agile was a “bottom-up, top-down mismatch.” How do we start to take that apart and make sure that there's not a mismatch but some sort of meeting of the minds? ~ Tom Cagley, SPaMCast 

That's a fantastic question, Tom. Yes, there is a type mismatch organizationally between agile, where agility is an aspirational concept, and process improvement methods like CMMI, which are operational in nature. On the one hand, you have engineers at the lowest level of the organization trying to push values uphill. And on the other you have the C-suite pushing down the process improvement Model without getting the team to own it.

So we have this type mismatch of aspirational versus operational, and I'll tell you, Tom, it ain’t pretty!



Let me give you an example. The agile manifesto guides us to adopt certain values. Those values, as we all know, are to collaborate with our customers, to have openness, to have courage, to have trust in our organization, to be iterative and incremental. These are values that companies need to adopt. But here’s the problem: We're seeing these values being adopted at the team level, whereas they really need to be adopted in the C-suite. The CEOs, CIOs, CTOs of companies should adopt the values and drive them down throughout the organization so that the culture of the company adopts those values. But that's not what we're seeing. We're seeing it being adopted at the lowest levels of the company – and they are trying to push those values uphill. That’s not sustainable.

Conversely, process improvement methods like CMMI, which are operational in nature not aspirational, are being driven from the C-suite, and not being driven at the lowest part of the organization where the operational activities take place.

This is our great challenge. As an industry, organizationally, we are inverted. This adds tons of overhead and unneeded activity. To fix it we have to start working with our executive teams to start, not on agile, not on CMMI, but on values. We have to start working with them to help them really understand that it's values that drive everything in the company, and values are way more than a poster on the wall.

Our values go right to the core of why we do what we do, and what kind of company we want to be.  That's why it's so important for the C-suite to get this right.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Friday, August 22, 2014

SPaMCast Question #2: If the large adopters continue to change agile, will agile become just a shell of a word?

[NOTE: Over the coming weeks, the CMMI Appraiser will be sharing snippets from a recent conversation with Tom Cagley on SPaMCast about whether agile is resilient – i.e., whether it will be able to spring back into shape after being bound or compressed by the pressures of development and support – and how frameworks like the CMMI can be used to make agile more resilient. Listen to the full interview at SPaMCast 296.] 

Jeff, If the large adopters continue to change agile, will agile become just a shell of a word? ~ Tom Cagley, SPaMCast

Tom, that's exactly right. And it won’t be the first time something like this has happened. There once was a day when Waterfall was considered the new, cool thing, and everybody loved it. Life was so simple then!



I am old enough to remember when people were saying, "Let's do this really cool idea, where we can make projects predictable. We can make people more productive. We can get everybody to understand and we can collaborate.” All of these words were being used back in the 1970s and 80s. Unfortunately, large adopters of the method have turned Waterfall into a shell of an idea. If this continues, there's no reason that agile won't go the same way.

I'm a big fan of history. I believe very strongly that history continues to repeat itself over and over and over again, and always has and always will. And for the science fiction fans among us, you saw this on Battlestar Galactica, where they always said over and over again, "What's happened before will happen again." 

This problem of taking a great concept and turning into a shell has all happened before. It will all happen again. And the same thing will happen with agile unless we take steps now to strengthen and make it more resilient.

What is resilience, anyway? The formal definition says resilience is power or ability to return to the original form or position after being bent, compressed, or stretched; elasticity. It is also described as the ability to recover readily from illness, depression, adversity or the like; buoyancy.

I believe these are two very apt definitions of resilience, given what is happening in our market and the influence of some of the newer players – such as the federal government, and big defense contractors – who are demanding that their vendors “go agile.”

In my opinion, this why we need a resilient model.

Check the Broadsword web site (www.broadswordsolutions.com) for our next webinar on Agile Resiliency.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Wednesday, August 20, 2014

Register for "CMMI - Everything You NEED to Know!"

Dear Readers,

Coming soon to screens near you!   If you have been tasked with, or interested in, transforming your organization into a high-performing, lean, and productive team, Broadsword is hosting another FREE Webinar for engineering and software professionals like you, on Friday, August 21, 2014 at 1PM EST.  We hope you can join us.



Watching this Webinar and learning everything about CMMI is an excellent choice for anyone who needs to get a grasp on improving, changing and elevating performance. At the deepest level, the CMMI provides you with an understanding of the way your company behaves, so that you can build better products, win new business and retain the customers you have.

That’s the promise of CMMI. The Model is all about the transformation of the culture of your company. It’s about improving and changing the way your company behaves, so that you create an environment in which the organization can manage its uniqueness in a structured way

So, whether you have been told you need to achieve a so-called “CMMI Certification” or you have been working with CMMI for years, you’ll want to check out the Webinar. You are sure to pick up some new ideas to help you get better at what you are ALREADY doing.


Like this blog? Forward to your nearest engineering or software exec! 

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students. 

Monday, August 18, 2014

Why you MUST involve the right stakeholders

[Dear Readers, for the past several months, our good friend Pat O’Toole, CMMI expert and seasoned consultant, has been collaborating with us on a monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month Jeff reveals why it critical to identify and involve the right stakeholders. Take it away, Jeff! ~ the CMMI Appraiser]


We’re an agile team and everything we do is “face-to-face.”  You might say we “specialize in collaboration.”  Why should we care about “GP2.7: Identify and Involve Relevant Stakeholders?”  It’s seems like defining a process for this doesn’t deliver any value.

I agree, you shouldn’t care about GP2.7. You should REALLY care about it!

The entire premise of “agile” is predicated on strong collaboration, transparency, and, most of all, being engaged.  So the real question is, why would you even THINK this wasn’t important?
Perhaps the words are getting in the way. The CMMI, rivaled only by a Piers Anthony novel for its esoteric lingo, may not say it in an “agile way,” but the authors meant pretty-much the same thing you do.  Another way to say it might be:

“We need engagement, and where the heck are they?”

Years ago, when I was a CIO of a software company, members of our development team came into my office complaining about team members not participating in important project events:

“He never comes to the meetings we put on his calendar!”

“She plays with her email during every design review!”

“The business customer never shows up!”

“The other developers don’t even LOOK at the code before a code review!”

That sounds horrible!  And we were using Scrum too.  THIS WASN’T SUPPOSED TO HAPPEN! 

Their complaints required some additional investigation.  So I asked them, “how big a problem is this?”  and “which meetings were worse than others?”

Crickets.

Don’t get me wrong, they thought it was a BIG problem, and it REALLY annoyed them, they just couldn’t tell me how big it was.  Or how small, or how much risk it introduced into the project.  Nothing.

Or as I like to call it, whining.  Wah wah wah!

By now my radar was starting to overheat.  “Wait, agile is all about engagement, but you’re saying people are not engaged?” Ouch.  My brain was hurting!

Agile teams are usually pretty good at engaging in “ceremonies” that are defined by Scrum (daily standups, sprint demos, sprint planning, etc), but are similar to traditional projects when it comes to engaging with the other  – no less important – events like design reviews, code reviews, testing, and other valuable, but not specifically “agile,” events.

Those of you who follow my blog know that I’ve been writing a lot about Agile Values, and that without them adopting “agile” is bound to end up being a frustrating experience.  This is because agile is really a set of values, not a method or set of techniques.  Agile techniques, like planning poker for instance, were designed in direct support of the values, not the other way around.  In other words, there should be direct traceability between agile values, agile methods and agile techniques. Holding daily-standups doesn’t make you “agile” unless your organization values failing-fast, transparency, and collaboration.  If they punish you for uncovering risks and issues every day, those daily standups will go the way of the buffalo right quick!

So, there I was, listening to my agile team members complaining about their own team … not being agile. Ugh. 

At the time I didn’t know much about GP2.7, in fact I had never even heard of it.  But I did know that in order to understand the risks of lack of engagement I needed information, so I decided to do something about it.   GP2.7 is, after all, all about risk.

A customer I had been working with had been playing around with an idea called “TeamScore” to keep track of stakeholder attendance for status meetings.  I thought that since “agile” is ALL about engagement, why not apply TeamScore to all events (as identified by the project’s defined process) and use it as a primary indicator of agile project health?  If engagement is a pre-requisite for agile projects, it seemed like understanding this data was a pre-requisite for running a successful agile organization.  It turned out to be a good guess that paid dividends well beyond the investment.

We first implemented Release One – an estimate vs. actual record of attendance at all ceremonies and events.  This revealed three facts:
  1.  At most of the events the attendance was not what was planned.
  2.  That wasn’t all bad, because we discovered that teams often clogged up people’s calendars with meetings they didn’t need to attend (which explains why some of them were not showing up).
  3. Some of the people attending didn’t need to be there, but they showed up anyway and ignored everyone (hey, it beats working!).

As soon as we started posting the scores in public team members came out of the woodwork! Those who were supposed to be there started showing up, and those whose attendance was superfluous self-selected out, removing waste from the value chain and increasing the bandwidth of the organization.

SCORE!  Instant improvement!

Having successfully spiked TeamScore in Release One, Release Two of our little experiment introduced a multiplier for engagement at each event.  If someone was unprepared, played with their phone the entire meeting, or constantly vanished to make a call (you know who you are!), we applied it to their portion of the average.  We didn’t report on individual scores, only the TeamScore. This introduced other ways for us to collaboratively set expectations, save money, and reduce our opportunity cost.  All because of GP2.7!

So learn to love it.  It can make the difference between saying your agile, and being agile.

© Copyright 2014: Process Assessment, Consulting & Training and Broadsword Solutions

“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton.  Please contact the authors at pact.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed.  New questions are also welcomed!

Thursday, August 14, 2014

What are the warning signs that our CMMI program will fail?

Dear CMMI Appraiser – I really got a lot out of your Webinar last month, “Everything You Need to Know about CMMI.” For my boss, if we decide to take on a CMMI adoption, can you repeat the warning signs we need to watch out for? ~ Rajib D.

Dear Rajib,

Thanks for following up from the Webinar. You know, the question of CMMI adoption is a serious one – but that doesn’t mean we can’t have a little fun with the answer! In honor of David Letterman’s decision to retire next year, I’d like to answer your question with a Top 10 Countdown, CMMI-style!

Here are the Top 10 Warning Signs that your CMMI adoption will fail:


Top 10 Warning Signs that your CMMI adoption will fail:

10. You tell your team to go “get a level” by Tuesday
9. You say, “We need to go right to Level Five (or Four, or Three)
8. Immediately after achieving Level Two, you say, “How soon can we get to Level Three?”
7. You look for a consultant to “do CMMI” to you
6. Your so-called CMMI consultant says, “The CMMI Institute makes you do it.”
5. Your so-called CMMI consultant says, “The CMMI makes you do it.”
4. Your so-called CMMI consultant talks about “implementing” or “complying with” CMMI
3. No one has any idea why you’re “doing CMMI,” but you’re doing it anyway
2. No one can articulate what the company is trying to achieve with the CMMI

… And the Number One warning sign that your adoption is destined to fail …

1. You’ve bought a tool that promises “CMMI in six months or less”

There you go ladies and gentlemen!  Don’t go away. We’ll have more fun right here for you when we come back.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy,performance innovation , software process improvement and running a successful CMMI program.

Tuesday, August 12, 2014

We want to improve as engineers and as a company – where do we start?

Dear CMMI Appraiser,  my boss told me to put a CMMI Appraisal Team together for our first CMMI appraisal, and am looking for good a CMMI Training course to help us improve as engineers and as a company. Where do we start? ~ Sam C.

Sam, welcome to CMMI! It’s great that you know that your goal is to improve, both individually and organizationally. What better place to start?



A lot of beginners come to the CMMI thinking they need to “get a CMMI level” or “achieve a CMMI Maturity Rating” – but that’s not really the point. Yes, you’ll achieve those results with a proper adoption of the CMMI. But it’s far better for your company and your career to approach the CMMI as a tool that can help you improve and change the way your company behaves, so that you can get better at what you’re ALREADY doing. 

Now, as you know, the first step in your CMMI journey is CMMI training. There are good CMMI Training courses available through the CMMI Institute. You can also take a CMMI Training class with a CMMI Institute Partner, like Broadsword (www.broadswordsolutions.com).

The advantage of taking the CMMI Training with a CMMI Institute Partner like Broadsword is pretty clear cut. We are real-world practitioners of CMMI. We use real-life stories, examples, exercises, and case studies as we go. We even cover dynamic concepts like CMMI and agile, and the cutting edge approach of agileCMMI that we pioneered, which can help you accelerate your career.

I'd love to have you and your Appraisal Team in our upcoming CMMI training class, Sam. It's always a pleasure to have participants who are motivated to adopt CMMI for the right reasons.

Register HERE for “Introduction to CMMI-DEV,” September 24-26, 2014, in St. Louis, MO.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Monday, August 11, 2014

SPaMCast Question #1: Why is it important to toughen up agile?

[NOTE: Over the coming weeks, the CMMI Appraiser will be sharing snippets from a recent conversation with Tom Cagley on SPaMCast about whether agile is resilient – i.e., whether it will be able to spring back into shape after being bound or compressed by the pressures of development and support – and how frameworks like the CMMI can be used to make agile more resilient. Listen to the full interview at SPaMCast 296.

Jeff, why is it important to toughen up agile? ~ Tom Cagley, SPaMCast 

In my work with engineering and software professionals from a wide variety of companies, large and small, in industries like aerospace, defense, finance, transportation, energy, and manufacturing, I see a lot of the same problems repeated over and over that can be eliminated by strengthening agile or "toughening it up," as you say.


But let's take a step back for a second.  One thing I started to notice a few years ago was that agile was really starting to exponentially multiply across many of my clients. As a huge advocate of Scrum and XP, I am really interested in how it delivers better software. This widespread adoption of agile is a good thing that we all want to see.

But there’s something going on with these companies that’s not so good. Even though the adoption of agile is multiplying rapidly across the industry, it is doing so in a very low-level way. The adoption of agile has been broad, but not tall.

In other words, project teams all over the world are adopting agile, but their management is not engaged. Their management is not understanding why agile is a good thing. So while organizations everywhere are adopting agile techniques, the businesses themselves are not changing to be agile.

This is a problem. In some areas of the company, there are software teams using Scrum and XP and other methods. In other areas of companies, they are not using those methods. Instead, they are using business methods that are in conflict with agility.

Why does this matter? A lot of big, large-scale adopters – such as the DOD, the federal government sectors and many of the big contractors – are starting to request more projects be run using agile, because they have heard it's a good thing. But if you take a look at their procurement or marketing and sales or senior management, they have no concept of why agility is a good idea.

They haven't figured out what it is, but they think they want it. So they give their suppliers mixed messages, such as: "We want you to be agile. But instead of doing this daily standup thing, can't you do it once a month? Because you're wasting my time by making me come to this. And can't you give that Microsoft Project work plan with 900 lines in it? All I ask is that you plan this project out. And be agile.”

So, while we hear these large adopters saying, "We want you to be agile," what they are really saying is: "We want you to work faster and cheaper. Truthfully, we don't really understand what this agile thing is, but we want you to be it.”

This is a problem. Organizations like this are driving change into the agile community. Unless we get serious about making agile resilient, the changes they impose will be detrimental to the future of agile.

For those who would like to know more about toughening up agile and making agile resilient, we invite you to sign up for our August 7, 2014 Webinar: Agile Resiliency: Scaling Agile so that it Thrives & Survives.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Sunday, August 3, 2014

Who can be CMMI certified, an individual or a company?

Hey, CMMI Appraiser, we’re confused about CMMI certification. Who gets CMMI certified, an individual or a company? ~ Lou F.

Hey, Lou – until recently, my answer would have been, “neither.” That’s because, until recently, the CMMI Institute has not used the phrase “CMMI certified,” “CMMI certificate” or “CMMI certification.” This term is commonly used in "certification environments" (companies who must earn and keep various ISO and/or US Government certifications).  In this case, CMMI is sometimes thought of as "just another certification." We have always thought that focusing solely on certification is a misguided interpretation of what actually needs to be done, which is to drive the transformation of the culture of your company.

So while companies don't get "certified" in CMMI (they get "rated"), there has been a change on the individual front. Finally, individuals CAN now be CMMI certified!



The CMMI Institute has recently announced a new certification: the Certified CMMI AssociateTM. This certification helps individuals committed to excellence in process improvement to identify themselves for professional career growth and advancement. The CMMI Associate certification also allows organizations to be confident in hiring credible practitioners.

There are thousands of job listings that list "CMMI" as a desirable skill, and this certification can be used to help match applicants with hiring organizations, much like the PMP or CSM certifications do. That’s what’s cool about the CMMI Associate certification – it helps both individuals and the organizations that hire them. The CMMI Associate certification is a win-win.

For individuals – the CMMI Associate certification provides confirmation of your knowledge of basic and intermediate concepts of CMMI. When you are a certified CMMI Associate, you are demonstrating your dedication to elevating organizational performance.

For organizations – the CMMI Associate certification helps organizations that are looking to hire experienced employees and partners.  You will be able to rely on the new accreditation to help identify those who are best qualified to help you cultivate and enhance internal strategies for process improvement.

How does it work?

You are eligible to apply to become a CMMI Associate after completed an official offering of the Introduction to CMMI course or the Fundamentals of CMMI elearning course. Once you have completed the course work, you can take the CMMI Associate exam. The 30 question, 1-hour examination is composed of multiple choice, true/false, and multiple select questions.

How much does it cost?

The CMMI Institute offers the CMMI Associate exam for $250.  In order to meet the training pre-requisite,  you can attend one of our "Introduction to CMMI" training courses in cities around the country, and we'll even provide a discount code that allows you to take the exam for less.

For more information about the CMMI Associate role and requirements, please contact us at info@broadswordsolutions.com.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Monday, July 28, 2014

What would happen if an extra small company got a CMMI rating and won a government contract?

Dear CMMI Appraiser – we are a small Northern Virginia company of 7 engineers that designs the best wide area surveillance equipment on the market. Because we’re tired of missing out on government contracts to huge companies like Northrop Grumman, we are looking into getting a CMMI rating. But if we do get a CMMI rating and win some of these contracts, would we need to start acting like a Northrop Grumman? ~ Doc F

Dear Doc,

Not just “no” … HECK no! Your strength as a small business is in your agility and your ability to quickly adjust and adapt to your customers’ needs. Don’t change that! The CMMI is a scalable model that was designed to be molded around your business goals and objectives. It works the way you work, regardless of the size of the organization. The only limitation you face is your own imagination.


In terms of company size, Doc, bigger is not necessarily better. There is nothing about large companies like Northrop Grumman that makes them inherently “better” at delivering the type of engineering solutions that you do. Likewise, there is nothing about the CMMI that requires the infrastructure that only a large company can afford. Those are myths that sprang up due to a lack of credible information available to small companies.

Not to say you are unaffected by the history that customers have with the big companies. Large buyers like the federal government have come to expect this type of framework from their suppliers. By adopting the CMMI, you won’t be doing what they do. You’ll be doing what YOU do! (Only better.)

One of the ways you’ll be better with the CMMI is you will be able to anticipate your clients’ needs in a predictable way. Large-scale buyers like the federal government or automotive companies anticipate their need for this. One of the ways they are able to differentiate among the most qualified suppliers is to use CMMI as a tool that, in its simplest form, provides a model for how great product and service companies perform – that’s why you see so many large companies adopting the CMMI.

But that doesn’t mean every small company needs to act like Northrop Grumman. No, my friend, that's not advisable or even possible. Instead, adopt the CMMI with a goal of becoming a better version of you!

For more information about how the CMMI works for companies with fewer than 20 people, feel free to visit www.cmmixs.com, and download our white paper, “Shattering the Myths about CMMI and Extra Small Companies."  You'll see why it's better to act like the company you are than one you'll never be.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Monday, July 14, 2014

How much bidirectional requirements traceability is enough to satisfy REQM SP1.4, and do we have to include both vertical and horizontal traceability?

[Dear Readers, our good friend Pat O’Toole, CMMI expert and seasoned consultant, is collaborating with us on a new monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month Pat talks about bidirectional requirements traceability. Take it away, Pat! ~ the CMMI Appraiser]

Requirements Management (REQM) SP1.4, the practice that focuses on bidirectional traceability of requirements, is like the obnoxious sibling that demands to be the center of everyone's attention, to the detriment of that very special child who is much quieter and certainly much better behaved.  In the case of REQM, the well-behaved child is SP1.5 - Ensure Alignment Between Project Work and Requirements. So let’s pause for a moment and give that angelic child the attention she so rightly deserves…

There are essentially two ways for things to get out of alignment with requirements.  First, since most of us are human, every once in a while we make mistakes. Perhaps the designs/test cases don't cover a requirement or two, and perhaps they include a design element/test case that isn't directly tied to any of the requirements – thereby representing defects of both omission and commission. Typically such issues are detected through peer reviews or some other verification technique.  To rectify such issues, the designs/test cases are simply corrected or otherwise knocked back into alignment with the requirements.

The second case occurs when everything is in glorious alignment with the requirements (cue the harp), but then that blasted requirement change is accepted.  Given the change, something now has to be realigned with this updated set of requirements.

The specific goal supported by these sibling practices is, “Requirements are managed and inconsistencies with project plans and work products are identified.”  That latter half of this goal statement – the bit in bold – is the “glass half empty” view of the SP1.5 practice statement: “Ensure that project plans and work products remain aligned with the requirements.

So here’s the punch line – although SP1.4’s expectation of “bidirectional traceability” gets all the attention and, with its discussion of “horizontal and vertical traceability,” more than its share of angst, it is merely the ENABLER of SP1.5 – the “maintain alignment” practice.  The thinking is that by establishing such traceability, the engineers are much more likely to cover all the requirements in the first place or, if not, to have their peers use the traceability mechanism to uncover errors of omission and commission when reviewing their work products.  In addition, bi-directional traceability enables more efficient analysis of candidate change requests, as well as more effective realignment of any and all affected work products with the new set of requirements.  And THAT’s why the model suggests we implement traceability – it’s simply a tool to help us keep things aligned.

And which project work products should be kept aligned with the requirements?  Absolutely EVERYTHING – after all, if it weren’t for the requirements we wouldn’t have a project!  So the project plan, schedule, issues log, risk list, emails, use cases, prototypes, design elements, code, test cases, deployment plans, etc. etc. should all be targeted at meeting the project requirements.  However, although everything the project team does should be focused squarely on satisfying the requirements, not all of the work products they generate will gain efficiencies by being traceable to them.  Which ones do?  Ah, now THAT depends!

So if you only focus on the obnoxious problem child, you may establish a bi-directional requirements traceability mechanism so intricate and academically beautiful that it warrants a patent, but one that may not best serve its intended purpose.  The engineers, who abhor doing non-value-added, administratively burdensome busy work, may begrudgingly use the thing, but their hearts won’t be in it.

On the other hand, if you encourage the engineers to exercise professional judgment by establishing mechanisms that ensure that the key work products stay aligned with the requirements, they’ll get it, they’ll build it and, more importantly, they’ll USE it!  I don’t know about you, but I would much rather have smart engineers do smart things to help themselves than to force them to do something they don’t want to do just because some model tells them that it’s good for them – whether they believe it or not.  Remember – when it comes to engineers, improvement is best done with them and for them, not to them!

© Copyright 2014: Process Assessment, Consulting & Training and Broadsword Solutions

“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton.  Please contact the authors at pact.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed.  New questions are also welcomed! 

Tuesday, June 24, 2014

Where can I get CMMI Training in Virginia?

Hey, CMMI Appraiser – I need PDUs towards my PMP Certification, and I want to learn about CMMI.  I understand I can get them by taking one of your CMMI training classes. Where can we get CMMI training in Virginia? I am looking for an introductory level course. ~ Bonnie P.

Hey, Bonnie – we have a CMMI training class in Northern Virginia on July 16-18. As of this posting, there are only a few slots open. But before you rush to sign up, I’m going to ask you to do a little self-reflecting.



The Introduction to CMMI training class is designed for software and engineering professionals who are interested learning about CMMI, process models, and how to use them to be a great company.

It’s true that you can earn 21 PDUs towards your PMP Certification (or 2.5 CEUs) while learning to improve software and engineering performance with the CMMI. But keep in mind, the reason engineering and software executives participate in CMMI training is because they are looking for ways to make their companies better. Whether it is software improvement, finance, product development, marketing or HR, they can use their CMMI training to make immediate, lasting improvements in their companies.

Taking an Introduction to CMMI training course is an excellent choice for anyone who is tasked with, or interested in, transforming their organization into a high-performing, lean, and productive team. If this is your intention then, yes, sign up for CMMI training today:

INTRODUCTION TO CMMI TRAINING
Click here to register for: the Introduction to CMMI training in the Washington, DC area

By reading this post, you have already experienced a taste of the biggest different between this CMMI Training and the other guy’s. We help you learn to use the CMMI to set the right goals and objectives, and keep asking the right questions, starting with “Why are you doing this in the first place?”

With learning as your goal, you’ll stay on the path to greatness, and external reward, such as earning PDUs, and achieving a Maturity Level of the CMMI, will be just byproducts of your journey.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.