…unless we create voices for a radical change, the critical IT component of our acquisition system will be swept into the same bucket as weapon systems which are currently pushing toward more oversight…
As with all organizational activity, resource allocation is the most obvious power base (“he with the gold rules”) and the place where large amounts of organizational energy are applied. As a public organization that has commanded a large percentage of the Federal budget each year, the DoD expends a large amount of resources (people, policy, and oversight) to provide public transparency to DoD expenditures in order to assure Congress and the tax payers that public monies are spent wisely and properly. With the exception of a few well publicized examples, the Department has done an credible job of ensuring that public money is properly spent.
…our adversaries are exploiting most modern IT capabilities to support command & control, sensor information aggregation, and weapon control.
The DoDD 5000 is the cumulative result of energy spent to ensure that public monies are properly spent in the acquisition of DoD equipment and services. As argued in my previous post, defense IT acquisition challenges…, February 13 2009, although the DODD 5000 policy and oversight processes are working well to ensure that public monies are not improperly spent, there is a large chasm between “not improperly spent” and and wisely spent to achieve the needed result. In the case of DoD acquisition expenditures directed at the acquisition of information infrastructure I suggest that we are properly spending resources according to policy and procedures and failing to achieve an effective return on tax payer investment.
Arguments against the current process include:
-
Not timely;
-
Too much oversight;
-
Long development and procurement timelines;
-
Poor fielded functionality;
-
Difficult to maintain; and,
-
Long-term contractor dependencies.
Every new administration, as far back as my career spans (and that is a long time:-)), has vowed to clean up the DODD 5000 acquisition process. In turn, each Administration has:
1. Rewritten the DODD 5000 policy documents to modify philosophy, processes, and oversight activity;
2. Enforced the new policy with commensurate oversight process adjustments;
3. Improved acquisition workforce capability through training and billet allocation;
4. Modified contract philosophy between hard nosed fixed price contracting to more flexible cost plus contracting or some combination in between; and,
5. Placed more or less responsibility for management and delivery upon the contractor.
Although some might claim that the above adjustments have created useful outcomes, none have addressed the uniqueness and criticality of IT systems. To date we remain saddled with an ineffective IT acquisition and oversight process not valued by government, contractors, or the Congress.
For the Obama administration the concern over acquisition has become a primary National Security goal. In a recent Washington Post Article, Hill Panel to Begin Review of Defense Acquisition System, Monday, March 9, 2009; Page A13, staff writer Steve Vogel states:
“Last week, President Obama ordered a government-wide review of federal contracting procedures. In pledging to get a handle on defense contracting, Obama cited a Government Accountability Office study last year that found $295 billion in cost overruns among major Pentagon weapons programs.
This ‘wasteful spending,’ Obama noted, comes from investments in unproven technologies, lack of oversight and ‘influence peddling’.”
In a separate March 5th Aerospace Daily article John M. Doyle Obama Supports Acquisition Reform Bill, we see the preliminary suggestion of added bureaucratic oversight… once again:
“At a White House news conference, Obama praised the defense acquisition reform bill introduced by Sens. Carl Levin (D-Mich.) and John McCain (R-Ariz.), the chairman and ranking member of the Senate Armed Services Committee (Aerospace DAILY, Feb. 25)… …Among the reforms in their proposed Weapons Systems Acquisition Reform Act, one would require the Pentagon to conduct preliminary design reviews before giving approval to new acquisition programs.”
So once again we are headed toward a new round of DODD 5000 reforms and possibly a new law governing weapons systems acquisition. As in the past, unless we create a voice for a radical change, the critical IT component of our acquisition system will be swept into the same bucket as weapon systems which are pushing toward more oversight and slower acquisition time lines.
…I recommend that we parse this problem into at least two distinct and separate acquisition domains…
Rather than working to embed my thoughts into this “change” ritual to overhaul the macro DoDD 5000, I recommend that we parse this problem into at least two distinct and separate acquisition domains. The two primary domains would be: (1) weapon platforms & weapon systems; and, (2) information infrastructure and information systems/applications. My rationale for breaking acquisition into these two domains is driven by the distinctly different pace of technology change between the two. In the case of the weapons systems domain, technology change is less driven by rapid commercial technology change and acquired systems generally are fielded and sustained for 2-3 decades with engineering changes capable of supporting the needed evolution through system life cycles.
The information systems domain, on the other hand, has been tightly coupled to commercial information technology, with its 12-18 month technology evolution, since the early 1980s. As the DoD information systems moved from purpose built computer systems to commercial-off-the-shelf information systems our National Security became directly related to the advance of commercial IT. On the network front, DoD spent large amounts of money working to invent networking technology that would satisfy its various mission system needs, only to forgo those developments in favor of the DARPA invented Internet protocol based networks that are the heart of today’s commercial Internet revolution.
This is all of the good news. It is these commercial technologies that have helped us deliver precision weapons across large distances with stunning accuracy and effect. The bad news is that the very same technologies that the US is leveraging are available to our adversaries. Furthermore, our adversaries are not hampered by an acquisition system designed to ensure that billion dollar weapon systems are wisely spending tax payer monies. It is now evident that our ability to deliver rapid information technology change to the battlefield could represent the difference between success or failure in future military confrontations.
One doesn’t have to examine enemy tactics in recent attacks to know that our adversaries are exploiting most modern IT capabilities to support command & control, sensor information aggregation, and weapon control. One of the significant obstacles to early success in the Iraqi conflict has been the IED (Improvised Explosive Device) usually triggered with some version of information technology. The recent terrorist attack on Mumbai India demonstrated the ability of a small force to control a city through modern mobile phone, satellite phone, and computer systems enabled command & control. It will be egregious if we allow our adversaries to leverage commercial IT to defeat our own decision processes while improving their own.
The next post will suggest a practical and incremental way that we can demonstrate an alternate acquisition process for IT system capability.
Marv, I salute what you’re doing here. The national security related information dominance of our country depends on how we procure these systems all of which are “changing faster than we can buy them”! I would hope that this takes on some legs. It should not be an iterative change, but a determined cultural shift. There are many lessons from our past about how we’ve missed opportunities to “see and communicate information to warfighters” which should form a nice basis for understanding the directions you’ve outlined. Many of us on your distribution list probably feel the same. I hope we’ll all support the direction you’ve outlined. Bill Owens
The pith of your insight is in the last section of your great article. Get it up front so more will read it. Yes! Exactly why Mother’s sent cell phones along with their home-baked brownies to the front in the early days of the Iraq confrontation. That was quickly made illegal. Hmmmm. How long does it take us to learn???? Lois
Marv, Spot on as ever. Re: accountability to spend “wisely”… an issue associated with acquiring information systems and sub-systems vs. platform or kinetic systems is choice of appropriate metrics. IMHO we need to bake your ideas about information system acquisition “goodness” into objective, useful, enforceable, parameters that get measured in the trenches and rolled up into executive dash boards.
Operational Availability (Ao) is a good Key Performance Parameter (KPP) for defining traditional “readiness” for many reasons. It focuses on a measure of effectiveness (MOE) that is important operationally, namely % of time available for useful activity. It is objective. It provides an enforceable basis to interpret policy. It provides options to allow PMs to manage program risk in pursuit of delivering the required readiness. Options regarding policy, cost, architecture, or standards are the means to the end, not the goal per se.
On the other hand, Ao has limited application in an enterprise information processing system because not only is it system-centric rather than capability-centric, it is based on an abstraction appropriate for hardware rather than software-intensive environments. So…In context with its concepts of “netcentricity” and the “Global Information Grid” (GIG), JCIDS introduced the Interoperability KPP and then replaced it with the even more abstract Net-Ready KPP (NR-KPP) . While the JCS high level net-ready policy goals are clear, NR-KPP engineering level “compliance” is not defined objectively or definitively. Therefore, there is no standard, repeatable, approach to testing or certifying “availability” with respect to “net-readiness” like there is to certify “Operational Availability” with respect to traditional system readiness.
To modify Ao in context with information system acquisition, we should consider that the overall objective of the GIG is to enable “information superiority”—i.e. processing information faster and better than the bad guys. Hence, the GIG aims to be a software-intensive system-of-systems that delivers continuously improving capability across a distributed enterprise. Run-time availability is not the only measure critical to GIG readiness. The adversary has access to the latest information processing capability available on the commercial market. So achieving information superiority requires not only shorter decision-cycles in run-time, but also shorter capability delivery cycles in build time. Accordingly, MOE that assure net-ready availability must address build-time availability as well as run-time availability. Further, simple run-time availability of clean data streams, i.e. the traditional QoS of a network, does not assure information superiority. Achieving information superiority requires availability of valued information and the non-availability of distracting low value data.
IMHO, we need an information-value-based approach to address these realities. We need to quantify “value” by defining MOE in context with operator-defined specific needs for measurably better operational outcomes as follows:
• Process-level MOE address build-time availability of resources in context with new capability delivered in time to provide measurable competitive advantage.
• System-level MOE address run-time availability of data streams in context with better decisions made in time to provide measurable competitive advantage.
• Policy interpretations objectively define, e.g., Intellectual Property Rights (IPR), need-to-protect vs. need-to-share, open modular architecture, commercial best practice, and data strategy in ways that provide measurable competitive advantage.
Marv,
Great to see you are still focused at the major problem that has hampered the IT/Command Control mission area. Wouldn’t it be great if we could get an acquisition capability like our advesaries….and keep the necessary dollars to ensure delivery and keep up with tech refresh..
Great blog and I am in your camp.
Very much on-point in that as long as the regulations governing weapons systems and information systems are bundled together one or the other will be badly served by inappropriate metrics or unrealistic expectations. We need a clear, logical separation and MOEs based on the needs and dynamics of each domain.
Thanks Phil, and it doesn’t stop with the DoD… this needs to extend into all of our government IT spending…
Good blog article and comment by Chris Gunderson.
“Value of a pound of information” has always been hard to quantify — perhaps the opportunity of responding to the new administration’s acquisition reform initiatives could enable program assessment & portfolio management communities to take a round turn on how they assess and manage programs and intiatives that produce information products. We need to recognize that past efforts have not been very successful, however (see “Looking Back” at the foot of this post).
One issue we (who work DoD information program requirements, material solution development, experimentation, and the co-evolution of doctrine & TTP with material solutions) need to address is governance. The natural champion for separating IT acquisition reform from weapon system acquisition reform is OASD Networks & Information Infrastructure (NII). They are bogged down dealing with large intractable issues (e.g., Joint Tactical Radio System and satellite communication systems delays and cost growth). There are forward-looking folks in OASD Acquisition, Technology, & Logistics (AT&L) and OASD AT&L Director of Defense Research & Engineering (DDRE) who could help move us forward; but they are not main-stream in the DoD acquisition community.
A key to success, in my opinion, will be the selection of people who will staff Secretary of the Navy and Ass’t Secretary for Research, Development, and Acquisition (ASN RDA). These folks will have to deal with the big issues of planning & executing a viable Navy shipbuilding program. Will they have the time and interest to advocate separating information program policy & governance from weapon system policy & governance? There are folks within the existing ASN RDA organization who could be very effective, if given leadership support. One step that could be taken would be to form an ASN and Chief of Naval Operations initiative to restructure Program Element lines from the cold-war legacy information programs to more broad and flexible Program Element structures that recognize the need for frequent update of the program goals. If ASN and CNO were to take that on, they will have to sell the idea to Congressional staff, who often see changes in P.E. lines in terms of opportunity to reallocate funding.
“Looking Back” —- We’ve been talking about decoupling IT policy & governance from weapon system policy & governance for a long time: Marv explained this to me in the 1980’s. The Navy Copernicus initiative tried to motivate this change in the 1990’s, with little success that I’m aware of. Global Information Grid was another effort to make the change in policy and governance: I’m not confident that the changes that have been introduced have had a positive result in terms of delivering more capability to Operational Level of War and Tactical Level of War operators.
Thanks Chris… see my comments to Brian about Max Weber and charisma:-)
Marv,
Excellent observations and recommendation. I would add that the IP centric environment in which we now operate is characterized by numerous complex interrelationships and interdependencies, yet we are hamstrung by these cumbersome requirements validation and acquisition processes tailored to large weapons systems and platforms. As you note, this repeatedly results in delivery of information technology as it approaches or even passes obsolescence. For organizations tasked to acquire, field, and sustain coherent solutions that span multiple layers in the OSI stack, there is an additional burden of multiple, disparate requirements and governance chains for different facets of the solution set, further challenging our ability to deliver coherent, integrated and synchronized capabilities.
Alan, amen… we need to spend our valuable time resolving the complex interrelationships rather than the artificial “oversighting” processes that are slowing us down more than is warranted.
I’va always believed in first tackling those things we can do successfully to show contempency and understanding of the problem.
In the world of IT the DoD lacks a very important qualification and that is a Network Mission Essential Task List. This is true for every Service and DoD writ large. Lacking this, from the unit level thru Joint Combined Operations, we lack a common understanding of the language, level of readiness and capability, from the smallest tactical network to the GIG.
Let’s get each Service to begin to define their list of Network METL’s then move to Joint Network METL’s. After all, we have METL’s for every single phase and depth of warfare except Information Warfare.
Just a thought, not a sermon
Thanks Jay, good idea. Like our DODAF architecture efforts, the issue will be adding discipline without it becoming just one more bureaucratic layer with a team of over sighters validating that someone has completed this step…
I completely agree with the basic premise, but I think another aspect of the problem should be included…we continue to think about IT system in the way we did 20 years ago – as big stovepipes of functionality that require huge investment over long periods. Because of this, we set up long critical review processes to ensure we make the right selection, and monitor the progress over time with large, complex governance. Unfortunately, for 100 reasons, most big system run into problems, overruns, schedule slips – functionality gets reduced or cut, limited functional solutions get fielded as ‘good enough’ and we start planning the ‘recompete’ or next replacement system. Its an vicious, expensive cycle.
Part of rethinking our acquisition strategy should be rethinking our architecture strategy. While SOA is a step in the right direction, it should not about making our stovepipes service-enabled – it should be about designing our systems to be assembled from a set of components. Many of those components may be services, but they don’t need to be – they only need to provide the functionality required for that component and meet the interface specification.
The key is to look at from a hardware manufacturing perspective. Today, we have lots of examples of hardware that has been componentized very effectively. We see this at the computer board level, at the computer box level, in our cars and buildings. I don’t need to buy a whole building from vendor X. I can buy the doors from X, the windows from Y – having building standards allows any component meeting the standard to be used. In a similar way, we have software component standards. We could create more if we need them. But we should be defining our DoD-wide architecture as an assemblage of components – specifically assembled to provide the business process flows and functional capabilities required. The acquisition process for software should be finer grained, contracting at the component level, building out pockets of functionality component by component – no more locked in big stovepipes.
If we look at why commercial is evolving so fast, in part its because they don’t design $B systems and develop them in a rigorous way over years. They use mash-ups and components, cobbled together open source, dynamic business processes defined in higher level representations, frameworks for rapidly assembling information and processes. While you can argue the military needs a higher standard, I would argue the fundamental approach of component-based assembled solutions is sound and appropriate, even in a mission critical military environment.
I could go on, but hopefully that captures the basic concept. We need to think about IT different than procurement of a weapon system – and that includes, IMHO, rethinking how we design and build systems to take advantage of the state-of-the-practices in software: components, patterns, services, abstractions, dynamic processes, flexible middleware, open standards and reconfigurable frameworks. That, combined with the other elements Marv presented, could give us a solid strategy for success.
Todd, great points and I completely agree. We need (and many are trying to create) the equivalent of the model T car parts model for IT. You also bring up a point that I will discuss in future post. That is the current 5000/Congressional processes force us to make every acquired capability a new program that starts, rises to the appropriate (by comptroller standards) funding hump, and then falls to a completion data. That start/stop paradigm has no basis in the IT reality of our world from my perspective. I like the enterprise paradigm better because I use a river as the analogy for enterprise. We can’t stop the river. We can slow down and work in the eddies but then we need to get back in the river is a way that supports and extends the river. Our engineering goal needs to be that we flow with the river, we don’t fight, diminish or in anyway kill the river.
Marv,
Nice to see your wise and thoughtful insight out here in the blogosphere.
Keep on, keepin’ on. Nice read.
Lori
Great post Marv, I’d be interested in seeing where you draw the line between weapon system and IT system, as that’s been a source of contention over the past few years. We moved to the argument that anything that moved 1’s and 0’s should be considered IT and back again, normally with programs trying to bin themselves where they thought oversight would be easier rather than any real analysis on how to do that-it’s been driven by OSD politics more than anything else. The interesting thing is that even saying “System X is a Weapon” is still often too coarse-grained, as there may be very logical break points even within a program. Case in point is what the sub community went through with their ARCI (Acoustic Rapid COTS Insertion) in the late 90’s. The requirement to add better processing and algorithms to the submairne sonar and fire control systems at a much faster rate was driven by a similar issue-we couldn’t keep up with the rate of improvements the Soviets were making at that point becasue of acquisition issues you point out. What they found when doing this was that from the trigger to the weapoon, there is really very little need or desire for change. A Fire Control system calcluates the firing solution. You want it to do that exactly the same way every time, and design changes are normally only driven by the introduction of a new weapon. Upstream of the trigger is where the innovation needed to happen, and where the traditional prime contracter/JCIDS/DoD5000 system was really breaking down when asked to innvate in the C2 side of the system. The beauty of ARCI was that a very traditional organization (PEO SUBS) was able to make this break in their thinking and allow what everyone sees as a weapon system to exist in this hybrid state. I expect we’ll see more and more of these types of situations as operators demand both more robustness in their IT systems and more innovation in their weapons systems.
Phil, great points and I agree with your division of labor. When I was at postgrad school (prior to the creation of the PC:-)) I had a picture i loved. It was a picture of our blue and white planet from space and the caption read: The world is analog, isn’t it? Since that time it has almost become a truism that we only revert to analog when moving something to and from us humans. So I think that 1s and 0s cannot be our discriminator. Rather, as Rick pointed out, we likely need to work the fully IT systems and then only carefully modify our building of the weapon control systems.
I would agree with Mr. Barber’s assessment of change.
“A key to success, in my opinion, will be the selection of people who will staff Secretary of the Navy and Ass’t Secretary for Research, Development, and Acquisition (ASN RDA).”
I have watched Navy create programs to invoke change; Copernicus, IT-21, FORCEnet, GIG, etc… In each case we have handed the vision of implementing these ideas back to the same acquisition workforce to implement, and each time they start with a bang, then fizzle as the traditional acquisition force begins to treat them like any other weapon system.
New programs don’t change people, new people change programs. The selection of people focused on understanding these issues is important, and then implementing these new ideas in policy so they can carry-on beyond administration boundaries is a critical step. I think we often bring in senior leaders that have the best of intent, but they are often left to work with the entrenched bureaucracy with survive administration change.
So in addition to Mr. Barber’s comment, it is equally important that we develop and sustain the workforce that will embrace and maintain the change. But this is often never accomplished, as we keep sending our folks to the standard acquisition pipeline training.
Brian, great point and I think smack on target. The reason I like the GIG-BE example so much is that it represents several components of this solution set. John Stenbit was able to effectively articulate the vision and sell it to the Congress and DoD to create the money in the short term. Then Tony Montemarano’s team implemented it within the short timeline established (and before John Stenbit left the system). An this was all done with a minimum of acquisition oversight. All of these things happened because of people, not because of process. So this leaves us with the dilemma. Can we create a new process that will enable good people to routinely succeed or are we stuck hoping that new examples of GIG-BE serendipity will emerge from time to time?
Interestingly, Max Weber, one of the pioneers of modern public administration theory had this to say about bureaucracy: “Although organisational bureaucratisation increases efficiency and the capability for greater production, that mechanical efficiency also threatens to dehumanise its participants. Weber also believed, however, that the only way people could make a significant contribution was to subjugate their personalities and desires to the impersonal goals and procedures of large scale organisations. Paradoxically, Weber believed that the only way to escape such a mechanical future was for a charismatic leader to transform the organisation into something new.”
So even Weber leaves us with the question of whether we will always need a charismatic leader to create a successful system. I prefer to believe that charismatic leaders help create the bureaucratic doors through which we can build momentum for good system implementation. When those doors are open we need to be able to move the new capability through the opening before the door closes. Had DISA not been able to push the GIG through the door that John Stenbit opened prior to his departure, we might still not have that capability…
“The two primary domains would be: (1) weapon platforms & weapon systems; and, (2) information infrastructure and information systems/applications. ”
Marv,
This thought is key. DoD 5000 has, forever, been focused on platforms. Information systems, to be of real value, cross platforms (and therefore programs). So a perfect platform acquisition doctrine would simply not give us any purchase on the information system problem.
Symptom. Within NavAir there are over a dozen ‘data links’ Each is associated with a particular flavor of aircraft. For example, The F/A-18, models a-d have one conforming to the Mil-Std-188-220B spec; the F/A-18, models e-g have Mil-Std-188-220C (and no, they’re not interoperable with each other).
Chris G has a solid thought in focusing on Ao as one of the requirements — it is, in my experience, the single most important requirement, and always the one to be dealt with _first_ when taking a concept and turning it into a deployed, supported capability. Good Ao analysis inevitably drives some other issues into the no-nevermind locker.
But that’s one level of abstraction too low … or to abuse a different metaphor, we need to peel at least one more layer off the onion. We don’t have the concept right in the acquisition system. So perfect execution is likely to be perfect execution of the wrong concept. So what concept do we need?
For information systems, the concept revolves around [modularity, interchangeable parts, interoperabiliy]. Note that in your recitation of DoD 5000 bullets, these don’t make the list. They never have; the lingua franca has always been [cost, performance, schedule]. Chris B comments that Copernicus was an attempt to change this and he’s right … indeed almost every Op92 initiative over the years attempted to change this … but we’ve never gotten a good modularization model. Copernicus had a good start in the software engineering area (concentrating on software modular component reuse) but didn’t get the comms modularization right (partly because the CSS program was the big moose blocking the path).
If the focus is on rewriting DoDI 5000.2, then the stroke of the pen changes are really quite minor. The existing instruction requires a half dozen DODAF views of each program manager.
– pick one of them. SV1 is the closest to what we need. This is where we write up the modularization model — with the intent that each program yield interchangeable parts that fit the model. It needs to be written and enforced at a level above the PMs because all PMs need to use the same modularization model. Otherwise one PM will produce legos and another erector sets and a third tinkertoys. So the SV1 needs to be furnished to PMs from higher in the command chain (this is what CIOs, by the law, are supposed to do but with one early exception I don’t see that that’s ever been done)
– the modularization model is quite simple and revolves around one objective: ‘extend the internet to mobile platforms’. KISS the commander’s intent here. (Chris and Chris will both weigh in with a ton of ‘yes, buts’, no doubt, but focus on the schwerepunkt and keep the number of syllables into two digits.)
– for info, I’ve found that the modularization model can be quite brief — two principles with five qualities for each principle — fingers only, no need to remove shoes.
– at the DoD/DoN/DoA level, enforce the modularization model on a few big ticket programs. The satcom and JTRS ones that Chris B has fingered are certainly eligible (GIG-BE is a poster child of getting it right in the terrestrial-WAN sector). Like the GCCS experience, once we get a couple programs on board, the bureaucratic difficulty slope will be downhill. Right now, ‘interoperability’ is a subject addressed in the checklist at milestone 2 or 3 … no, has to be at Milestone 0. 1) because at that stage it’s pretty cost neutral — costs as much to do wrong as to do right and 2) once the PM is past initial startup, you can’t change the modularization — it’s locked.
Now … you can get to the specific program requirements. If you look at the comms problems as ‘what do we need to do beyond what commercial Internet is doing’ (see Joel, chapter 3, verse 10), then the requirements analysis becomes quite tractable … with, as Chris G mentioned, Ao as the first diff on the list to be worked. I’ve found that the list of diffs only has four items on it (the other three are security, QoS control and reach to mobile platforms).
I do need to defend the point of view in one respect, else Chris G will again jump me on it. The view above is very comms/internet focused. And it largely ignores the end system requirements (applications). And Chris’ observation is correct. But:
– if you get the internetwork right, … just once … then you can start, restart, recycle, … any number of applications.
– requirements analysis in decision support systems seems to be doggone near impossible a priori. But if you be a v1 in the field, coupled with a good feedback loop and supported by a modular software design (along with a good software business model, like open source), you can feel your way into this space. You can’t do that with the comms system.
– one internetwork; many applications. Two different production philosophies, two different procurement approaches, two different acquisition and maintenance approaches. (we have NMCI to demonstrate)
… who won the docking pool?
Rex, I think you and I could come together on a new model here. We do need to treat the hardware intensive infrastructure IT a bit differently than the software intensive applications. That is why it is exciting that GIG-BE has demonstrated a successful rapid infrastructure IT example. We all know that we are working hard to create the cloud computer/SOA infrastructure layers on top of the network to help free up the application speed to market but we are currently lost in a morass of IA, standards, and competing implementations. We need to free this up and them move out. I will suggest some ways that we might do this in the next post.
Brian’s note and mine crossed in the mail … can’t resist a postscript. Brian is exactly right about the people. But there are some sly’s that we can be constructive about.
Chris B trotted out the usual catalog of poster children on the BAD side; I’d put GIG-BE up as the poster child for getting it right. What made that program tick? And, after all these years of lampooning DISA (remember DISArray, DISAster, etc?) why did they deliver the right thing in the right way?
First the _what_. GIG-BE was an up-gunning of the DISN backbone. DISN had gotten bits and pieces of baling wire and chewing gum over the years … and was a packet switched system implemented by bellheads. And DISN had been following the fee-for-service business model — the objective is not to gain warfighting power; the objective is to make the program pay for itself. We can all add to a list of what was wrong…
DISA identified ~90 POPs and interconnected them with OC192 (10.6Gbps) pipes. Does DISN need that kind of capacity? No. But by locking onto OC192, it channeled the program: there’s only one thing you can plug those firehoses into (a router) and there’s only a couple companies on the planet that build these sized backbone routers (Cisco, Juniper). 1) the program mimics what the commercial ISPs have been provisioning and 2) there was no R&D component in the GIG-BE budget. Pretty clear what ya gotta do? How can you NOT do the right thing?
Requirement. Each of the 90 POPs got at least two of these OC192 pipes (Ao requirement).
Non-requirements. The GIG-BE program had NO applications in the program. This keeps them out of the vertical-slice, artisan based, and non-interoperability holes. Since GIG-BE is purely a router-router interconnect problem (a WAN problem), all end systems and the apps they host are on the other side of a router somewhere … and outside program scope (this is a fatal for several satcom programs).
Execution. With this kind of conceptual setup, its actually quite hard for a PM to screw it up! The program’s POA&M slipped a little to the right … ho-hum, 2.5 years grew into 3 years.
Just to close the loop. If JTRS had been viewed as a radio-WAN program. With an objective of extending the internet to mobile platforms. … we wouldn’t have a decade-old program with no deliverables. Ditto for MUOS.
Rex, you have hit one of my hot buttons. As a person that has spent my share of time complaining at DISA, I believe that the GIG-BE team under Tony Martemarano’s leadership is one of the best examples of doing rapid IT acquisition for infrastructure. As you said, it was an iterative (but radical) upgrade to the old DISN, it used non working capital funding, and it opted for owning the light pipes which now provides us with enormous flexibility. The current crime with this system is that it is very under utilized because we have saddled the use of the pipes with working capital funding taxation rather than offering our warfighting nodes unlimited bandwidth as they should have. The fact that this was carried out in 3 years almost from the time of John Stenbit’s vision up to completion is unmatched by any other IT program I can think of. While one could argue with the architecture or implementation of this system, I believe it is proof that we can procure IT rapidly and achieve great results. This is the poster child for DISA. We need more of this kind of progress.
This set of ideas is very, very important. So I’d like to raise a minor complexity, in the hope that when somebody wants to take our advice, it’s as precise as needed. The complexity is that the two categories (weapon systems, information systems) have a high overlap area. That is, most weapon systems today include a large fraction of information system. So, probably, we need to embrace 3 categories and treat them appropriately, where the 3rd category is the IT components of a weapon system. I believe that this category ought to yield to agile methods when we define desired qualities and trade them off appropriately. That won’t be easy, but it will yield much better results, especially if tied to an iterative approach with reasonably short cycles.
Rick, I could have expected that you would catch and expose this issue. I thought about that overlap while putting this post together so am glad that you are addressing this overlap. I think if we pilot some change in the non overlap portion of IT that we can create a modified process that would address this overlap to support the IT agility that also needs to be applied here. As we continue to define a path forward that gains some support from Congress and our DoD/IC peers maybe we can better address this overlap area in parallel…
Marv,
Right on. Our information warfare quiver cannot be subjected to the same “forever” development process that is used for ships and airplanes. Open architectures, COTS and concepts such as SOA set a stage for rapid updating/upgrading of systems that we need to exploit to ensure we stay on the leading edge of IT advances.
I have advocated “availability” over “reliability” for a long time. Was really happy to read Chris Gunderson’s comments on Ao. In C2 systems getting five nines of reliability only requires paralleling five 90% reliable components (My ORSA degree showing through). What matters to the Commander is having the system work when he/she needs it. That is availability. And availability needs to be 1.0. Losing a vital piece of information or inability to communicate a command .001 percent of the time could still be a disaster if the outage time coincided with a serious event.
Another area outside of the acquisition that I believe requires some innovative engineering is in the field of IA. It seems to be an adage that the defense will always lag the offense in the information security space. Again, the time lag from seeing an intrusion to defending against it could prove fatal in certain scenarios. This is obviously a tough one but one that is gaining in importance day by day as we see the increasing attacks on our systems.
Keep up the great work Marv. Maybe someone will listen and we can make IT the warfighting resource it really needs to be.
Dick, great comments. I would like to attack the IA issue as a component of this larger issue but as you suggest, we are waaaaay behind on this front. I believe we need a completely new paradigm on how we deal with security but that is for another time, but not to distant.
Marv,
Thanks for another great thought-piece. I see example after example of why we must decouple processes in the way you spell out here. I also see example after example of senior leaders who think they should not be decoupled. There is a disconnect in thinking on this topic and my belief is that your writing can help folks sort out the right approach.
As an enterprise IT guy, I have no doubt, we will be buying commodity IT to meet our needs long into the future, so the capabilities of IT, including its functionality and security, are going to be dependent on advances in the commercial sector. DoD acquisition guidelines and security specks can influence commodity IT, but only if they are well thought out and can align with/contribute to commercial industry.
For an example of a government led IT activity that holds great promise of influencing commodity IT see: http://www.nsa.gov/ia/programs/suiteb_cryptography/index.shtml
That link points to info on the NSA “Suite B” activity.
Activities like this one should help us accelerate the decoupling of IT acquisition processes, I think.
Bob, good inputs. I will learn more about these and look forward to you help on this needed progress.
Cool Blog. Change can set us free. Here are some of my insane ramblings. Have mercy Remember I spent a lot of time in Iceland, a bankrupt nation.
Near term IT investment: Investments like NMCI and CJTMK have not worked out like we hoped. I contend that NMCI has reduced our overall IT agility and stifled innovation while making us less secure from malicious attacks. We are so concerned about IA that we live our IT lives like the boy in the bubble. I say let the virus’s come. It will make us stronger in the long term. Diversity is a good think. MAC OSX, Linux, BSD, Open source, I say bring-em on. It will be ugly at first but people will get smarter, our robustness will increase over time and we will see new innovations happen much faster. Try to count the number of cool new applications for the iPhone. Why do we block access to sites like iTunes, Google Apps and YouTube? Innovation begets innovation and needs to be encouraged and rewarded.
Long term IT investment: To me the problem is bigger than DoD5000 or issues with JCIDS. Why as a nation are we willing to spend billions on weapons systems, like F22 and LLT, while neglecting real investment in IT. We insist on maintaining separate networks for security while paying lip service to “black core”. Our idea for MLS is to think in terms of separation kernels and guards that maintain the status quo instead of building true trusted operating systems that are capabilities based. Has anybody seen a true secure OS being developed or even discussed in the US? I don’t mean separation kernels, of which there are several. We are still sending official messages in all capital letters. We should engage smart IT people that understand fundamental computer science. I recommend we try to engage Dr. Shapiro, Dr. Kiselyov, Dr. Tanenbaum and Dr. Torvalds about these ideas. Check out this exchange http://www.coyotos.org/docs/misc/linus-rebuttal.html. Where are conversations like this happening in the DoD?
Dave, good out of the box ideas as I always expect from you. Your revolutionary approach is good on many fronts. I am choosing to work the system because we always move incrementally, partly because of our shared power model of government, and partly because the many voices of a democracy mediate us toward a middle ground. If we were to fall off a cliff for some reason (an IO attack might help there) then revolution like you are suggesting could be a stronger possibility. Perhaps we could create your world in an alternate environment like the DREN…
Marv, great seeing you at the Navy NRO Conference on 19 Mar.
Excellent review along with comments from others on why its a good idea to separate IT acquisition reform from platform/”stuff” acquisition. Certainly makes a lot of sense for the IC to lead here given the IT centricity of intelligence as a discipline. More importantly relative to DoD the IC represents a reasonable test bed for the changes you recommend that should scale to DoD. One thing the IC does not have to wait for reform to do ——- shift to enterprise wide IT buying where the issue points to an enterprise solution. ICD 501 is a good policy start here joemaz
Joe, good points. I think the IC has generally been ahead and more disciplined than the rest of DoD on IT systems. Let’s continue to collaborate on how to push this noodle forward:-)
“…come together on a new model here. We do need to treat the hardware intensive infrastructure IT a bit differently than the software intensive applications.”
This is really fairly easy if you play with the problem. Most of the ‘hardware’ is in the wide area networks (e.g. satellites). Further, the wide area networks are the part of the internet that must be application-agnostic because they traffic in an indefinitely large number of applications. The modularization principle here is straightfoward:
I. all comms comes in the form of routable networks.
This principle applies to both terrestrial and radio WANs.
Within the radio-WAN routable network cloud we need to pay attention to four details:
II.
A. stable media access control
B. ability to multicast (called point-to-multipoint by IEEE 802)
C. some infrastructure protection, appropriate to the segment
D. an SNMP agent that allows fault management.
The end systems that attach to the network are similarly easily configured … and agnostic to requirement. “All end systems (the sense, decide and act nodes of an information system) attach to a LAN”. The meta-purpose is that this modularizes the end systems away from the WAN. To implement this principle, we need 5 things:
1. ethernet interface
2. a packaging mechanism (start with e-mail)
3. digitally sign all emitted data
4. set the differential services code point
5. SNMP agent in the box … same reasoning as above.
The way out of the stovepipe bureaucracy is illustrated by GIG-BE — none of the end system issues are part of the program; only the terrestrial WAN ones. That’s what needs to be institutionalized.
BTW, Tony Montemoreno was indeed the PM for GIG-BE, but I’m told the conceptual brains behind was Dave Mihelcic.
Now, at the risk of letting too many cats out of too many bags… both USMC and USA are heading down the same road. If you hold up the IEEE 802.16 standard to the radio-WAN desiderata above, you get a perfect fit. And both services have figured that out … or at least a conspiracy of captains and majors have. DoD has over 1000 nodes today. Both programs are largely stealth: the USMC program has a nonsensical acronym ‘WPPL’ attached — what’s important is that there’s no ‘R’ in it … so we don’t disturb the JTRS dragon. Both programs are progressing … either with off-budget resources or with maintenance money from existing programs.
How to institutionalize at the DoD 5000 level. The modularization model recited above should be handed to each PM (perhaps in the form of an SV1). Each program (and this applies to platform-oriented programs too) complies with the modularization model. (Contrasted with the current approach of requiring each PM to create own). properly policed this would provide the discipline desired … without meddling in the specific requirements of each information system or component thereof.
“differently than the software intensive applications. ”
Note that the model is largely orthogonal to this aspect. But there are two (both beneficial) observations to be made:
1. The ‘software intensive’ parts are in well-defined modules — the end systems — that are easily life-cycle-maintained.
2. most of the software intensiveness seems to occur in the decide module in the end systems. Here we have an additional observation, the 85% one. 85% of the functionality required by tactical decision support system A is also required by tactical decision support system B. And is wholly mission independent. The Navy discovered this in in 1990 when there were two dozen tactical DSS programs underway in SPAWAR alone … each with a PM dutifully operating within own swim lane. … go look at how the open source operating system distributions work.
Marv,
I worry about this particular solution because everyone is currently trying to carve off “their” area as “special.” I believe nearly everyone accepts the current system has all the problems you have identified which have not been solved by many very good political appointees who have tried. In thinking about this over the last couple of years, I have concluded that the answer lies not in the process, but in the talent of the individuals who are doing acquistion — not in the talent of the politicals, but in the talent of the military people the Services assign to acqusition.
I believe that nearly any problem can be solved by the application of sufficient talent. Concurrently, we are not going to make a lasting difference in acquisiton by subdividing one area off and “saving it.” We need to force the Service Chiefs to put in sufficient talent to address their long-standing issues and be able to solve their own problems day-to-day. Specifically, we need five changes:
1. An Acquistion qualified 4-star in each service (USMC 3-star deputy) with the associated staff,
2. Double the number of acquisition flags,
3. Double the number of acquistion SESs,
4. Manage acquisiton civilian careers (training, promotion and duty assignment) the same way as officers (by AT&L)
5. Establish a system of Acqusition “mentors.”
Best regards,
Dave
Dave, Thanks for taking time to add your thoughts to this discussion. As you know, I always value your ideas. I take your point about carving off pieces as a non solution in the larger sense. I hadn’t thought about the ideas you have added (adding more talent to the system) because I see a lot of people around me toiling at the onerous processes we have institutionalized under our DODD 5000 banner. Having grown up in the AEGIS Wayne E Meyer program office I do understand the value of a strong program structure, strong mentoring, and many rungs on the learning ladder before one reaches the top of the system. In reflecting back on that experience I can relate that your suggestions would help push us closer to having more talent to create those special program offices.
That said, I can’t help but feel that we are spending too much money on excessive oversight for our acquisition. I could apply my model and ideas to the entire system but my focus has been on IT and C4I so I wasn’t trying to bite off the full problem. I will mull on this further and see how this feels a couple of days down the road:-)
Marv,
Much of the debate for oversight and execution of MAIS as a separate stovepipe is documented in GAO reports, Grace and Packard commision. GAO reports documents history of DoD directives and policies for execution and oversight and implementation.
1. I would suggest reading ” TIdes of Reform ” by Paul Light for larger context of governement reform.
2. Services and DoD, suffer from NIH and are not willing to give up equity. Also system is not set up to incentiveze OSD staffers to gain “buy-in” with services. It is too easy to opt our of exercising “checks and balance” and results in argument by position of authority. Leads to strong centralized decision making ( dangerous). No “wisdom of the crowds” here.
3. Current Acquisition talent pool has a tough time delineating execution vs oversight and at what levels this should occur. Also what took five years to tear down will take three times to build back up. Current system will find it hard to compete for “millenial ” genration talent. We will have to appeal to ” service to government” and provide good on board and off board experiences so we can catch them again later in their career.
Probably want to look at how we provide good experiences for younger generation coming up. Emphasis should be place on leadership pipeline ascension ( instilling good values) vice throwing more SESer/Flags at the problem.
4. Do believe there is merit in creating “Esprit de Corps” and reaching a “tipping point” / going viral to infect the rest of the Acq professional community.
5. Suggest your forum gain wider audience in DAU forum.
Gary,
Great comments and a useful discussion on the human resource component of this issue. I believe that we will see much more of an inflow and outflow between industry (at least DoD related industry) and government which I view as generally better than the golden handcuffs structure that the older retirement system created. I think it is still true that young people can get more responsibility at an earlier age in government than industry which is a good reason for them to spend time in government at a young age and then move out for mid career years, and possibly back in during late mid career years. All good reasons for this to be worked harder.
Marv,
Thanks for including me in your mail group. I am flattered. I would like to remind the group of some recent events that provide additional urgency to the subject of this blog. Both SECDEF and SECVA, as well as the President now, have announced their committment to create a single, life-long, virtual eRecord for every service member/veteran, spanning their lives from entry onto active duty to internment. In addition, both Secretaries have committed to SOA as the strategy.
I offer to the group that this has now created a “macro-use case”, if you will, from which we now need to derive subsidiary use cases, business models, the abstracted SOA meta-models, and allocate work to build these services to DoD and VA.
While all posts above are quite thought provoking, those of Todd Carrico, Ph.D., resonate most with me WRT this new direction set by the President and both Secretaries. To attempt to expand pon Todd’s comments (not sure i am qualified to do so, but here goes anyway…) it might be useful to think about information resource management from the perspective of the Federal Enterprise Architecture (FEA). The FEA is simple in concept, and not to difficult to use to explain IT investments to senior leaders, showing graphically the linkages (“value chains” or “mission threads”) that start at the very top (the PRM) and extend to the very bottom (the TRM), with connections through the BRM, SRM, and DRM.
We certainly need to move away form platform-based thinking, and the stove-pipes thereby created. In addition, we need to move toward resourcing and building common services (really common, i.e., with multiple consumers) with tight control of interfaces.
VA and DoD have now stood up the InterAgency Program Office to facilitate this work. Architecture (as a process and a product) will be a significant responsibility for the IPO.
I would appreciate your thoughts. If we think this through together, we can come up with a really strong concept of IRM suitable to the challenges now laid in front of us.
Tnx…
PT
Paul,
Thanks for the good comments and for raising the IQ of this great group of thinkers. I agree that Todd has it mostly right and because he and you are commenting on the rapidly changing capability and agility being created by new technologies (primarily virtualized computing, more network ubiquity, and service based software that relieves the burden of creating application foundation code). I believe that it is this technology change that will overshadow our current outdated oversight practices and allow us to more quickly build and field new functional capabilities in future DoD IT systems. What I would like to see happen, while these technologies are maturing, is for DoD to begin to pilot some rapid IT capabilities. Further to your and Todd’s points, is that fact that we are quickly getting to an era where the acquisition professionals like ourselves will spend the majority of our efforts building layers of enabling infrastructure tools which allow operational users to create the applications that best meet their needs, not unlike you and I build powerpoint slides using the Microsoft or other slide builder tools. The hardest nut to crack for this rapid user interaction is the security challenges.
Marv, Your insights are right on target, which is why your contribution to the IT Acquisition Advisory Council will be invaluable. After reading both recent DSB Reports (IT Acquisition and Netcentric), my view of that the Defense Industrial Base is unable to fix itself is reinforced. Since the signing of the Clinger Cohen Act, which required agencies to leverage COTS and associated industry implementation best practices, I have seen little real change within either OSD ATL or OSD NII processes. They are still using MilSpec approaches designed for weapon systems platforms (with 20 year lifecycles) to buy IT infrastructure, business systems and information sharing solutions. The Services seems to know this, but unable to change the mind set of pentagon bureaucracy who hide behind their processes and are not vested in the success of the programs they oversee. Over the past 10, since taking the helm of the Interoperability Clearinghouse, I have seen huge resistance to change within the acquisition community, with few note worthy exceptions where leadership at the top forced change (AF, BTA).
Four years, one of the smartest acquisition guys I have met, David Tillotson, gave ICH an opportunity to rewrite the IT pre-acquisition process, making the Clinger Cohen Act actionable, via a project called the AF Solution Assessment Process (ASAP). Thanks to support from Mr. Wynne, Gen Peterson, Chuck Rieckers, Keith Seaman and Mr. Tillotson, we were able to overcome the many rice bowls and cultural impediments, proving that there is a better way. Cycle times for requirements alignment, capability determination and solution assessment were reduced from 18 months down to under 45 days. The formula for success was simple, dedicated leadership and a shared commitment to better outcomes. ASAP was applied 10 times with the same outcomes; actionable requirements, measurable capabilities, objective technology assessment, and an automated business case analysis approach.
Since then, this Method is being replicated for Navy CANES, USMC TRACOM, JFCOM MNIS, and BTA CAE, demonstrating a potential model for a new IT Acquisition Process. Thanks to Mr. Keith Seaman at BTA, this models is now being institutionalized with formal documentation, use cases and modeling tools that overcome the process gaps of JCIDS and the DODAF.
Now back to the DSB Acquisition report which recommends a separate swim lane for IT Acquisition outside the DOD 5000, JCIDS and the DODAF. Do we have the leadership in DOD and the resolve to tackle this problem. Will Dr. Ash Carter’s reform efforts be undermined by the same rice bowls that have inhibited a dozen prior reform efforts. For real change to occur, we must reward risk taking and small failures. Our DoD leaders must support the mavericks who have been beaten down over the years. We must highlight the cost of maintaining the status quo, which is estimated to be over $20Billion per year in IT failures and cost overruns. We must holder our suppliers accountable with well defined Serve Level Agreements.
The US cannot dominate or win in Cyber space unless we fix these process and cultural problems. We must reward innovative thinkers and embrace our critics. We must embrace the innovators and change agents who have long been black balled in the past.
Marv,
For the rest of the crowd, here’s the link to the Defense Science Board Report mentioned by Mr. Weiler:
http://www.acq.osd.mil/dsb/reports/2009-04-IT_Acquisition.pdf
Our pre-acquisition, Maritime Fusion & Analysis Services (MFAS, part of Maritime Domain Awareness) went through Gate 1 Review today (a successful endorsement, BTW). In preparation, DASN C4I brought us up to see ASD NII about options to move forward on an AoA prior to Material Development Decision in order to 1) inform POM 12, and 2) get a head start on determining the way forward for MFAS, e.g., new MDA PoR, become part of an existing PoR, etc. As we all know, if we don’t have enough information to make cost analysis evaluation by this fall for POM 12, POM 14 will be the next real opportunity for a new start.
ASD NII made us aware of this pilot program for new IT acquisitions and we will be evaluating it now that we have approval to move forward with a pre-AoA or BCA activity. The 2 yr+18 month cycle of the pilot is attractive in comparison to what we would expect as a DoD 5000.02 MAIS but we have more studying to do on the pilot process.
Joe,
Thanks for the link to the new DSB report and for the example of the potential of MFAS to move faster than our normal acquisition cycle. It will be important for pilot programs like this to help pave the way for a new way to acquire IT capability. I will talk about other pilot options based upon new network infrastructure security technology work ongoing in Pacific Fleet in the next blog post.
Next week, DoD and industry thought leaders will be gathering at the Defense IT Acquisition Summit, Nov 12 in Wash DC to lay out the roadmap on how OSD will address the new NDAA directive to establish a separate IT Acquisition process. Secretary Bill Lynn will be kicking off the session with his perspective on how OSD will tackle this long running challenge. For years, the OSD leadership has tried to establish a “one size fits all” approach with the DOD 5000.02, making incremental changes every couple of years. Though designed to be a flexible process, the overseers never really wanted anyone to skip any steps unless signed off by DepSec.
My hope is that the new process does not fall prey to the NIH crowd who has prevented real change since 1996. If the rule of law is applied, especially the Clinger Cohen Act, the following success criteria would need to be established;
1) It must leverage real world industry best practices. BTW, the Defense Industrial Complex is not “industry” as defined by CCA. DoD need to learn from the successes and failures of multi-national giants like FedEx, Citigroup, Ford, BofA, GE to name a few who have a history of success.
2) It should take advantage of some of the agency innovative approaches that have delivered, like the AF Solution Assessment Process (ASAP), which completed the entire acquisition cycle in just four months.
3) It must follow OMB A119, that is to leverage existing process standards that have already proven to effective with IT. Not just standards for standards sake. For instance, OSD NII should recognize that the DODAF is a system engineering tool developed in the 80s for, you guessed, Weapon Systems documentation. After 25 years, it still lacks key elements including business view, performance metrics, solution view, and service component specifications. Sometimes, less is more, and maybe we should be looking at the OMB FEA reference models that more closely align with industry best practices.
4) The IT Acquisition process should not become another FFRDC product. Have we not learned from the “full employment act” methods like NESI, LISI, etc.
5) It should, as the DSB recommends, be modular and focused on delivering small, interoperable modules that have well defined interfaces in a services oriented context.
6) Also, as stated in the DSB, it should allow for continued user participation who understands the real trade offs and will likely help us all avoid “perfection” in leu of the 80% solution that can acquire at the “speed of need”.
7) And, the real litmus test will be, will the cost of doing the documentation exceed the cost of the solution. If so, it does not work.
My two cents. information on the Nov 12 Defense IT Acquisition Summit can be found at the new IT Acquisition Advisory Council web site; http://www.IT-AAC.org.
Cheers