Modernizing Application Modernization

The transformation to a digital business is causing many organizations to place a higher priority on modernizing legacy applications and replacing outdated packages with modern, adaptable solutions. Many organizations are faced with a difficult dilemma – how to modernize legacy applications that run the business when 75% of available funds are devoted to keeping these systems running and the processing they provide cannot be interrupted. In the past it was common to see modernization efforts take a back seat due to insufficient funds or lack of priority. Or, the efforts failed because the task was so complex that costs escalated uncontrollably. However, organizations are now finding that the legacy maintenance and enhancement cycle is not able to keep up with the pace of changes they face – be they from regulatory mandates or market pressures.  And, when combined with digital business pressures, modernization becomes a critical business need. 

IT organizations are now looking for ways to roll out new business capabilities to their customers and stakeholders that leverage the data embedded in the legacy systems.  However, these are hardened, mission-critical systems that provide the core processing for the business – and are resistant to change. As a result, they are recognizing that they will have to retire them and replace them with much more flexible IT capabilities.  The caveat is that this must be done in a manner that does not jeopardize the daily business operations the systems support.  

Too often, modernization has meant wholesale replacement via a lengthy, expensive and disruptive “big-bang approach which failed to meet expectations.  Today’s modernization approach is radically different from conventional application modernization which replicates yesterday’s architecture in a new technology. Today’s best practice:

·        Modularizes the functionality and delivers a set of services that map directly to the business model

·        Separates business rules from code

·        Uses enterprise-level Agile and Lean practices that allow rapid delivery and continuous improvement

·        Uses a factory framework that standardizes all nonfunctional and common code in a design platform that radically improves cost, quality and delivery time

·        Deploys as a series of incremental releases that may be vertical slices (areas of functionality) or horizontal slices (end to end processes for specific types of processing, such as application types, claim types, financial transaction types, etc.).  

Rather than converting code (where the resulting codebase represents the same tangled mess in another language), the approach replaces the legacy application with a layered, modularized architecture based on the principles of Service Oriented Architecture (SOA). In the past few years SOA has become the default architecture for modern applications to support the need for flexibility and continuous integration/deployment.

To achieve this, legacy applications must be understood from the perspective of the business and technical capabilities they provide.  Business rules must be harvested from legacy applications (or new ones derived where the requirements have changed).  Transition planning must define release points that include existing functionality (e.g., wrapped transactions) as well as new services.  During the transition, legacy interfaces must be preserved or replaced to maintain operational continuity.  Ultimately, the modernized application should reflect modern IT best practices and patterns, such as maximizing the configurability of the system via parameters (moving change points out of the code) where flexibility is desired.  Examples of these best practices include the implementation of software services, business rule management (rules engines) and business process management (BPM) tools. 

Everware-CBDI’s Service Oriented Application Modernization (SOAM™) methodology is a comprehensive approach that combines multiple leading edge, but proven, techniques to deliver flexible modernization.  Key features include: incremental evolution of capabilities, modular architectures to limit the impact of change, and separation of infrastructure platform and data stores from functional code so that each may evolve independent of the other. We combine the modularity of SOA and the rapid development and stakeholder feedback from agile development with automated techniques to efficiently produce, test, and deploy code. We have developed an end to end process that minimizes the gap between when business requirements are identified and when the functionality is deployed into production.  A key component of our approach is our project-configurable Agile Service Factory (ASF) which standardizes and accelerates the task of specifying services and automates the transformation of those specifications into fully-open, industry-standard code targeting popular platforms and frameworks.  Combining this with a DevOps capability closes the loop in rapidly delivering needed functionality.

The result of this approach for application modernization is the ability to incrementally modernize legacy systems that is rapid, priority based, and lower risk.  Ultimately, this approach enables the IT organization to move from a maintenance focus to continuous modernization.

The Role of EA in Federal IT Development

Dave Mayo, Everware-CBDI

January 2013

“There is more value created with overall alignment than with local excellence.”                                                                                                                                             –                                                              -- Don Reinertsen

As I look back on over a decade of EA in the federal government, I must say I am disappointed in the results so far.  In my experience as an EA practitioner, I have boiled down the role of EA to two areas:  (1) Support decision-making by the business (mission) and IT leadership – EA provides a repository of information about enterprise components and their interrelationships in the form of a set of models that can be queried to gain valuable insights; and (2) Provide guidance to IT to develop & deploy assets that support the business.  The role of EA here is to help eliminate (or at least reduce) redundancy, inconsistency, and inefficiency in IT and to promote alignment, sharing (reuse) and interoperability across the enterprise.  The first of these is an analytical role along the lines of business intelligence (BI), using the EA repository as the knowledgebase.  The second is a prescriptive role to evolve the portfolio of IT assets in the strategic direction of the enterprise (based on the EA Target Architecture).

Unfortunately, EA in practice seems to have come down to two completely different things: (1) A compliance exercise to meet the evaluation criteria of OMB and GAO.  Never mind the fact that both evaluation methods are intended to get agencies to undertake “real” EA.  Agencies seem intent on doing as little as possible to score well on OMB and GAO assessments.   (2) A descriptive exercise that typically focuses on the as-is situation.  Although, “this is what we look like” does have some value, it is limited and the exercise is very costly (ask DoD about the billion dollars spent on the Business Systems Modernization program).  But the primary shortcoming across many EA programs is the lack of the intention to be prescriptive, combined with the governance to make it happen.  It is the second role and shortcoming that I wish to address here – EA should provide the foundation for application development (AD) and modernization. 

The role of EA in AD lies in transforming abstract business requirements from the problem space into concrete solutions that address these requirements – while keeping all of the capabilities and solutions synchronized to provide traceability.  The most successful organizations have adopted the SOA architectural style and apply it at all levels of this transformation.  At the highest level, it involves depicting business requirements as capabilities - discrete units of process, data and technology that allow you to accomplish something of value.  Modeling capabilities (and their dependencies) is an important part of the Business Architecture because it starts the componentization thought-process.    As you move across the problem/solution divide, these capabilities are transformed into the services or solutions that implement them. 

EA has two roles in this transformation.  The first is to establish the rules and parameters to guide the transformation. This includes setting standards, similar to building codes, that must be adopted by development teams. These include standards for development patterns, semantics and deployment platforms, among others.  While constraining, these standards actually foster creativity and accelerate development because the teams don't have to reinvent/readdress these issues – in the same way that inventors of electrical appliances don’t have to worry about how to get electricity to the appliance.  The second role for EA deals with the big picture.  It is to produce the overall (target) architecture of solutions, and the services they consume, so that the development teams can build them out – similar to a “table-top” model of a building or a community that shows how everything fits together without providing the internal detail of each piece.  The overall solution architecture as well as the EA rules and constraints are provided to the acquisition process to be incorporated into contract language that directs the design and development of solutions. The solution architecture from the EA and the services consumed by the solution are provided for incorporation in the RFPs and define what the contractors are to build.

At the Solution Engineering phase of a program, EA should provide the definitional clarity described above (denoting the components of the architecture, their definitions, boundaries, and how they interoperate) as well as more detailed artifacts to guide design and development, including reference architectures and principles that identify patterns and platforms that can/must be used by design/development teams.  An example of a very successful architecture principle at a high level is that all functionality must be service based, that all interactions will occur through the service interfaces and that all service interfaces must be externalizable.  Amazon used this principle to implement all capabilities as services and in the process created a business platform to integrate all internal and external offerings.   (ref. Amazon;  see Steve Yegge’s Rant, 2011).

During design and development, the EA team should perform compliance reviews at appropriate stage gates to determine that the program is building out the components of the architecture and is complying with the rules and constraints.  Today, most EA programs review development progress “from an architectural perspective”.  But without a model to compare it to, the results are weak at best.  In many cases, development programs claim compliance with the EA because they can “map” their application to the Reference Models, rather than demonstrating that they are actually building out a part of a segment/domain architecture that will be reusable by others.

  

The bottom line is that AD should operate in a manner like most construction industries – develop an architecture and then build it, preferably by assembling solutions from pre-built components.  If the architecture is service based, it will be modifiable on an incremental basis and thus should stand the test of time.  If the solution components are specified and modeled in a rigorous manner, then automation (eg, code generation) can accelerate the process and facilitate the maintenance/enhancement process. 

This is the concept that we at Everware-CBDI have based our core competencies on.  We call it Service Oriented Application Modernization (SOAM) and it bridges EA and solution development.  SOAM consists of four disciplines: SOA, Agile Development, Model Driven Development, and Portfolio Transition Engineering.  This approach rapidly evolves the suite of IT assets to a set of rationalized, reusable components that can be reconfigured to address new challenges.   In the process we are able to align development results with the business goals and objectives. 

Agile IT: The Solution to the 25 Point Plan

In my last blog, I analyzed the 25 point plan for Federal IT reform from Vivek Kundra, inferring the four objectives for the Plan. In this blog, I outline the solution for achieving the objectives – something we at Everware-CBDI refer to as Service Oriented Application Modernization (SOAM). Recalling that the objectives of the Plan were to (1) increase IT cost effectiveness, (2) provide better IT support to the business/mission, (3) reduce development risk and cycletimes, and (4) reduce redundancy and inconsistency through reuse, an evolutionary modernization program is the necessary foundation. The era of large, all-or-nothing software development programs is over. In its place iterative, incremental, spiral, agile and other light methodologies are taking over. The goal is Agile IT – a portfolio of IT resources that not only aligns with business change, but is actually capable of enabling it.

Agile IT is an elusive term – especially in the Federal space – almost an oxymoron. However, there is an approach that we have applied successfully that contains the core components of Agile IT. Our SOAM combines best practices from Service Oriented Architecture (SOA), Model-Driven Development (MDD) and Agile Methods. At first glance, these might seem to be incongruous, even diametrically opposed, however, we have found that each provides a balance to the others and the result is highly synergistic.

For example, Agile Methods are adept at quickly creating software to serve specific purposes. However, Agile teams often begin with little or no formal documentation (eg, requirements or architecture) – the emphasis is on producing functionality (code) quickly that can be assessed by knowledgeable and empowered users. Therein lies the beauty of Agile: the solution evolves based on rapid feedback from users into the development process. But what if you don’t have access to knowledgeable users who are empowered to make decisions about the design for the entire organization (which is typically the case for government projects)? What’s more, Agile works best with small teams of highly-skilled developers who are well versed in the tools across the lifecycle and technology stack. The result is that Agile is more difficult to scale. You can’t just grow the team size – it becomes unwieldy. And if you have multiple teams, then you have communication/coordination issues. That is where SOA comes in.

SOA is an architectural style in which solutions to user needs are composed from services (imagine that!) that are accessed via well-defined interfaces. SOA allows the solution to be designed as a collection of interacting modules that can be reused in multiple ways. There are many benefits derived from SOA – such as lower cost from not having to build functionality (again), greater consistency in the enforcement of business rules, flexibility from the plug and play nature of services, and better alignment with the business needs. But for the development process, the key advantage is the ability to divide a solution into independent modules that are well insulated from each other. Obviously, you need other teams to integrate the modules into the solutions (à la “twin track” development); but, with an overarching service oriented architecture, the objectives of each team and their interrelationships should be clear.

Of course, with SOA the number of moving parts increases dramatically. Managing them is where MDD comes in. MDD uses models to document the requirements, design the solution and services, and generate the code or other executable artifacts (eg, BPEL, business rules, etc.) that become the system. Using MDD, the solution is developed at an abstract level and translated to more detailed versions. For example, typically a platform-independent version is created that is transformed to a platform-specific version by applying design patterns and platform conventions and constraints. From the platform specific models, the code is generated that become the executables of the system. In the Everware-CBDI MDD environment, some amount of code customization or extension is required (typically 10 -20%), but the generated code is intended for human consumption and hand modification. There are two key advantages from MDD in the development process. First, much of the functionality is generated – accelerating development, reducing the coding effort and associated errors, and improving the consistency of the code base. This also allows the developers to focus on customizing the code for the users’ requirements (value-added effort), rather than on the application “plumbing” that integrates all of the components of the solution. Secondly, MDD serves a coordinating role in synchronizing the efforts across all of the development teams. Changes in one module/model are automatically rippled across all related modules (avoiding the infamous ‘I fixed the error in 5 of the 6 places where it was found’ problem). This second benefit also implies more efficient maintenance for increased agility and a lower TCO of the software.

As you can see, at Everware-CBDI we have taken the agile approach a step further – developers don’t focus on producing commodity code (well, some of them do focus on specific code extensions); the focus is on refining the models from which code is generated. This is not only more efficient, it enables the rapid turnaround associated with agile methods to be combined with an architectural and engineering based approach to developing software solutions on a large scale. As, hopefully, should be clear from the above, each of these disciplines augments the others to make the approach so beneficial. The result is greatly improved productivity, flexibility of solutions, and responsiveness to the user needs – all of which support the four objectives of the 25 point plan mentioned above.

If you are interested in applying SOAM in your environment, please contact me (dmayo at everware-cbdi dot com) or check out our web site.

Dave Mayo, Everware-CBDI