Monday, January 28, 2013
Wednesday, June 8, 2011
Agile IT is an elusive term – especially in the Federal space – almost an oxymoron. However, there is an approach that we have applied successfully that contains the core components of Agile IT. Our SOAM combines best practices from Service Oriented Architecture (SOA), Model-Driven Development (MDD) and Agile Methods. At first glance, these might seem to be incongruous, even diametrically opposed, however, we have found that each provides a balance to the others and the result is highly synergistic.
For example, Agile Methods are adept at quickly creating software to serve specific purposes. However, Agile teams often begin with little or no formal documentation (eg, requirements or architecture) – the emphasis is on producing functionality (code) quickly that can be assessed by knowledgeable and empowered users. Therein lies the beauty of Agile: the solution evolves based on rapid feedback from users into the development process. But what if you don’t have access to knowledgeable users who are empowered to make decisions about the design for the entire organization (which is typically the case for government projects)? What’s more, Agile works best with small teams of highly-skilled developers who are well versed in the tools across the lifecycle and technology stack. The result is that Agile is more difficult to scale. You can’t just grow the team size – it becomes unwieldy. And if you have multiple teams, then you have communication/coordination issues. That is where SOA comes in.
SOA is an architectural style in which solutions to user needs are composed from services (imagine that!) that are accessed via well-defined interfaces. SOA allows the solution to be designed as a collection of interacting modules that can be reused in multiple ways. There are many benefits derived from SOA – such as lower cost from not having to build functionality (again), greater consistency in the enforcement of business rules, flexibility from the plug and play nature of services, and better alignment with the business needs. But for the development process, the key advantage is the ability to divide a solution into independent modules that are well insulated from each other. Obviously, you need other teams to integrate the modules into the solutions (à la “twin track” development); but, with an overarching service oriented architecture, the objectives of each team and their interrelationships should be clear.
Of course, with SOA the number of moving parts increases dramatically. Managing them is where MDD comes in. MDD uses models to document the requirements, design the solution and services, and generate the code or other executable artifacts (eg, BPEL, business rules, etc.) that become the system. Using MDD, the solution is developed at an abstract level and translated to more detailed versions. For example, typically a platform-independent version is created that is transformed to a platform-specific version by applying design patterns and platform conventions and constraints. From the platform specific models, the code is generated that become the executables of the system. In the Everware-CBDI MDD environment, some amount of code customization or extension is required (typically 10 -20%), but the generated code is intended for human consumption and hand modification. There are two key advantages from MDD in the development process. First, much of the functionality is generated – accelerating development, reducing the coding effort and associated errors, and improving the consistency of the code base. This also allows the developers to focus on customizing the code for the users’ requirements (value-added effort), rather than on the application “plumbing” that integrates all of the components of the solution. Secondly, MDD serves a coordinating role in synchronizing the efforts across all of the development teams. Changes in one module/model are automatically rippled across all related modules (avoiding the infamous ‘I fixed the error in 5 of the 6 places where it was found’ problem). This second benefit also implies more efficient maintenance for increased agility and a lower TCO of the software.
As you can see, at Everware-CBDI we have taken the agile approach a step further – developers don’t focus on producing commodity code (well, some of them do focus on specific code extensions); the focus is on refining the models from which code is generated. This is not only more efficient, it enables the rapid turnaround associated with agile methods to be combined with an architectural and engineering based approach to developing software solutions on a large scale. As, hopefully, should be clear from the above, each of these disciplines augments the others to make the approach so beneficial. The result is greatly improved productivity, flexibility of solutions, and responsiveness to the user needs – all of which support the four objectives of the 25 point plan mentioned above.
If you are interested in applying SOAM in your environment, please contact me (dmayo at everware-cbdi dot com) or check out our web site.
Dave Mayo, Everware-CBDI
Tuesday, February 1, 2011
Having been involved in Federal IT for almost 30 years and a contributor to many IT improvement initiatives, I am going to take a shot at reading between the lines and try to discern the goals and objectives underlying the 25 point plan. First, I will try to categorize the points into a few groups (assignment of points to categories is indicated in parentheses). Obviously, some of the points can be assigned to different or additional categories, but this is probably close enough.
1. Reduce the cost of IT infrastructure (1,2,3,20)
2. Modernize government IT acquisition and contracting (4,5,13,16,25)
3. Reduce IT risks through better program management (7,8,12)
4. Identify and implement IT best practices (9,10,11,14,24)
5. Promote shared services and modular development (6,15,17)
6. Align processes for capital investment, budgeting and modular development (17, 18, 19, 20)
7. Improve IT management and oversight (22,23)
This categorization, while useful, does not really tell us what the underlying goals are. For example, there is an emphasis on shared services and modular development – but why? So, the next step is to try to abstract out the principles and goals that drive the categories and points. But, before we get to that, let’s look at the major challenges facing the Federal government with respect to IT. Here are some of the most significant.
1. Major IT program failures – billions lost due to failed program cancelations
2. Inability of applications to interoperate or share data (due to technical and semantic issues)
3. Majority of IT budget spent on O&M, leaving little to fund development of new capabilities (resulting in a large backlog of enhancement and new functionality requests)
4. Development lifecycle takes too long – driving large contracts and large contractors; requirement change during development
5. Redundancy (same capability is built into many applications) and inconsistency (enforcement of the same policy differs in multiple applications)
6. Lots of money wasted on redundant infrastructure and redundant capabilities
Ok, so let’s get to the goals and objectives behind the 25 point plan. The bottom line is the need to rationalize and modernize Federal IT management. At a high level, this includes the following.
1. Make IT more cost effective (reduce the per unit cost of delivering capabilities)
2. Provide better support to the business/mission of government – enable IT agility to underscore business agility
3. Reduce the risk and cycletime associated with IT development programs
4. Reduce redundancy and improve consistency through reuse
Ok, so if this is what we want to achieve, how do we go about achieving it? Some of the ways include the following:
1. Rationalize IT platforms and infrastructure (virtualization, cloud)
2. Update the IT process (acquisition through deployment/maintenance, modular (twin-track) SDLC)
3. Improve roles and skillsets (CIOs, acquisition specialists, program managers, etc)
4. Improve communications and collaboration (government – industry, business – IT, providers – consumers, semantics, IPTs)
5. Institute evaluation and feedback mechanisms (Techstat process)
Which brings us back to the 25 point plan. Now that we understand what we are trying to achieve, let’s get on with it.
Dave Mayo, Everware-CBDI
Thursday, February 19, 2009
With all the talk these days about how “SOA is dead,” one might wonder: What does it really mean and what could have killed it? Yes, there has been disillusionment, but there are also examples of great success. From where I sit, I see the difference as a mismatch between SOA definitions and expectations. Let me explain.
There seems to be two major competing definitions of SOA floating out there. From the top down, the definition is that SOA is an architecture of interacting services. From the bottom up, the definition is that SOA is an integration technology – a more flexible way to hook things together (such as via web services). Each of these definitions has broad-reaching implications, from planning to governance to infrastructure support. From the business perspective the holy grail is the “adaptive enterprise” – the agility to turn the business on a dime to meet the requirements of a rapidly changing environment. The first definition of SOA holds out this promise – having a collection of architected services to mix and match allows a business to reconfigure the business and supporting IT to meet new demands. From a technology perspective, the second definition provides the benefit of network flexibility – making different technologies interoperable and allowing disparate technologies to be plugged in more easily.
So, for purposes of this discussion, the question is: is what is being promised the same as what is expected? In many cases of SOA disappointment, I contend the “buyers” (typically the business side) have been told that SOA leads to organizational agility. But, the SOA implementers have set out to deploy according to the second definition. Let’s be clear; there is a lot involved in implementing SOA according to either of these definitions. So, after the initial deliverables, the team shows the business what they have produced and the response is: How does this achieve organizational agility? The problem is they are both right – just using different definitions of SOA. Unfortunately, that is probably the end of the SOA initiative for that organization.
Perhaps what we need is to start each conversation with a comparison of the definition each party is using. This might help avoid the mismatch between what is expected and what is produced.
Tuesday, January 20, 2009
Forget IT. SOA can be applied to an organization (even one devoid of IT). If we throw out the conventional definition of organizations of hierarchies and functional units and replace it with a framework of business units offering services, we can derive a service oriented organization. Starting with the products and services we provide to external parties (external customers) and the products and services we obtain from external parties (external suppliers), we have the beginning of a framework to describe the organization. If we then add sets of internal customers and suppliers and define the act of providing a product or service to a customer as a service, we have the next step – a collection of services – tasks performed for a consumer. We can then map the incoming products and services to outgoing products and services by defining processes that consist of steps in transforming inputs to outputs. The processes and steps in the process are the services previously defined. So, processes become a string of services that are orchestrated to produce outputs. Now services are a lot like processes (very fractal) with inputs, activities and outputs; and, they can consume other services, so we need some rules to prevent service conflicts and circularities. This is achieved by defining an architecture of services with layers dividing the services based on rules about which services can call which other ones (for example, a typical rule is services at a given layer cannot call services at a higher layer).
Organizations that define themselves as collections of service (org) units, achieve agility from two basic facts: (1) processes can be changed by rearranging the services or replacing a given service with a different (new?) service and (2) new products can be produced by combining existing services in new ways and adding a few additional ones. Think about an insurance company offering a new coverage product – many of the activities required are the same as for existing products and can be reused.
In IT, the concepts are the same, except that now we are talking about software components instead of organizational units and they are even more flexible – software services can be offered to multiple consumers from the same distributed component on the network (i.e., the “cloud”). All of this adds up to a significant gain in agility both on the organizational and the IT side.