"SOAs are transforming traditional monolithic applications into services that allow flexibility, re-useability and on-demand access.”
It’s not often that you find as concise and succinct a definition of a technology in this industry as that, and Donald Mowbray of Oracle is to be given a hearty slap on the back for managing to cut through the jargon and hype to describe why SOA is such a hot topic.
Of course, it’s a more complicated concept than can fully be explored in one edition of the ITP Report, let alone one line, but it’s worth going back to the basics lest business lose sight of why its implementing an SOA solution in the first place. Like ITIL implementation, the goal is to improve the performance of a business’s IT systems and align and integrate them with the business, rather than solely installing the SOA itself, which is nothing unless you revise your business’s processes and organisational support.
How does SOA deployment enable organisations to run applications like a service?
Stan Alexander: SOA is a key enabler, but it is not as easy as simply deploying a set of software or announcing architectural standards. If you step back and consider the primary benefits you are seeking, such as business agility and better alignment of IT with the business, then you’ll see that moving from the traditional siloed application model to a shared services model is appropriate. To do so within your enterprise, you will deploy a services oriented architectural model, but at the same time you will be redefining and implementing your business processes and changing your organisation and culture in support.
Donald Mowbray: SOAs are transforming traditional monolithic applications into services that allow flexibility, reusability and on-demand access. The mindset that applications should reside on a dedicated server is shifting to an SOA-based model that creates a virtual and dynamic environment. We have seen the emergence of a set of important industry standards that are becoming available for developers to build interoperable services and composite applications, including JAX-WS, BPEL, WS-ReliableMessaging,WS-Addressing, SOAP with Attachments, MTOM, WS-Policy, UDDI, WSSecurity and Service Component Architecture. Oracle has helped to define these standards and is using them as the basic building blocks for the Fusion Middleware platform. Leveraging those standards to effectively create services out of existing applications, integrate them together at a data level and orchestrate business processes out of these services.
Enterprise applications are moving from user interface driven applications to assemblies of re-useable and interoperable services. These services represent simple business functions intended to be assembled together into new applications. One of the key advantages of this change in application architectures is that services can be rapidly reused in new and changing business processes.However, this approach to building composite applications and business processes doesn’t work without a standards-compliant platform for building services, such as that provided by Oracle Fusion Middleware.
Malhar Kamdar: IT is typically acquired or developed in response to the needs of a particular business segment and deployed solely for the benefit of that segment. Since IT tends to be funded and constructed by projects whose purpose is to address a given set of requirements, functionality tends to be duplicated. Traditional approaches to sharing functionality at a code or component level have failed due to this project-by-project focus of development activities.
A service-based approach to IT changes the way in which functionality is developed and delivered. Functionality is considered, factored and deployed once for use at all levels of the enterprise yielding the associated benefits of reduced cost, faster delivery, and IT responsiveness to the needs of change. In addition to requiring differences in the way that IT is funded and governed, a service-based approach requires a change in the way that functionality is packaged and deployed. SOA also considers the manner in which functionality is made available as services, and the way in which those services are managed and monitored.
Another aspect of traditional IT delivery is that each project typically chooses the most expedient method to satisfy its requirements. This leads to a proliferation of technologies that is problematic when deliberating how applications built on those technologies will exchange information. Previous attempts at standards-based component models like CORBA and DCOM suffered because of the technology required to execute them, slow development of supporting standards, or both. More recent developments like XML, Web services, and UDDI provide the foundation for a standards-based SOA that supports re-use, and for which the technology required to support the standards is readily available and truly platform agnostic.
Tom Bishop:By allowing applications, and even portions of applications, to be ‘wrappered’ by an SOA/WS interface layer and then publishing that interface in a well-defined place (registry), any application or portion of an application can be made available to any external third party just as that service interface. This is what enables organisations to publish and run applications like a service.
Maria Pardee: SOA deployment on its own does not enable organisations to run applications like a service. To run applications like a service you have to either develop the application as a service, or wrap the application so that elements of it behave like a service. For example, if I have a CRM system full of my customers data it won’t offer a service to find a customers address unless it already offers that as a service. Alternatively, I wrap the application with something like web services that enable me to query the customer address portion of the application or database and present that back to the calling application as a service.
How can enterprises better leverage SOA through the use of MDM, Grid Computing, and XML Networking?
Stan Alexander: The shared services environment naturally starts to put you in a shared data and shared infrastructure environment. Having a common taxonomy and definition of key entities such as customer, product, address, bill, etc. become essential to gaining the business efficiencies and flexibility you expected in your SOA project. Likewise, does it make sense to share a service but not network and servers? Some form of services grid or database grid can compliment your SOA both with facilitating distribution and with scalability. In addition, as occurred with SSL encryption (for example) you will begin to see repetitive, compute intensive tasks moved off the server and into optimised hardware devices. It would be a natural step to do your XML encryption or transformations this way.
Maria Pardee: MDM could be used to minimise the sources of data and provide a means of keeping data in step. In a SOA you might have developed a service that updates the address that you send a customers bill to supported on the CRM system. Your billing system might also hold this data, so if your service only updates the CRM entity then your MDM could be used to keep the billing data in sync with the CRM data.
Grid computing could be used to load balance the resources required to support services that you offer during peak load, but there are many other ways of doing the same in any architecture without using grid computing.
Since many services in a SOA are often based on XML based web services then XML networking may be of use. This is because it operates at the middleware layer and uses standard documents to integrate your applications. It also provides you with fully standard functions using languages for: message verification, XML translation, context based routing, high availability, data security, compression and caching and teh translation of application layer requests into standard web services.
Tom Bishop: All of these concepts (MDM, Grid, and XML Networking) depend on the idea of defining a set of services based on standard interfaces, putting those interface definitions and access rules in a well-known place, and making them available to third parties who only want to use the services through those interfaces, but otherwise don’t care or want to know about any of the underlying details. While the specifics of how MDM, Grid, and XML Networking all vary in their details and supported use cases, they each share this common theme, which is why SOA/WS is such a central concept to each of them.
Donald Mowbray: The biggest factor that is being overlooked is the actual control that organisations have on their enterprise resources. The ability to share applications across networks relies not only on the optimal software infrastructure where SOA provides the framework but also a more efficient control of enterprise resources than is typical in today’s computing environments. To execute an IT function, a process and a resource is required. Until these two are synchronised then the vision of services on demand will remain just that - a vision. This is where the benefits of grid computing and SOA converge. The point where grid computing and SOA join will unlock the full potential of these symbiotic technologies.
Where IT would have been concerned with the performance of applications on a system, now they must consider thousands of services all running simultaneously on a variety of hardware platforms. Grid computing provides the optimum foundation for SOA. By utilising grid technologies to create a high-performing and scaleable platform, organisations can build flexible and dynamic SOA-based applications for users that demand resources and new functionality on demand. The quality and availability of critical information is of the utmost importance to businesses. Most companies struggle with fragmented or out-of-date information, which can at best delay business decisions or, even worse, cause incorrect decision-making. Ubiquitous access to critical data improves customer service, provides competitive advantages, reduces legal exposure, and improves regulatory compliance, all of which directly benefit the bottom line, which is why MDM is a critical component of any SOA deployment. To improve information models and analytics, customers should create master data and shared domain models for data or registry-style hubs, in the form of Customer Data Integration (CDI) or Product Information Management (PIM), to improve data quality and acessibility.
Malhar Kamdar: Enterprise solutions such as MDM, grid or utility computing and the use of XML Networking technologies can contribute to increased efficiency in both the implementation of an SOA’s constituent services and to how its capabilities are surfaced at runtime. A well defined enterprise access services layer is a key feature of the SOA Reference Architecture of many organisations successfully implementing SOA. Combining that with the obvious notion that in most organisations enterprise software solutions are primarily designed to process data, the value of data in SOA is apparent. It follows that traditional concerns for architects building enterprise applications – such as data consistency, data quality and data accuracy – are equally applicable to architects involved in SOA activities. So, MDM has advantages for SOA adoption, since the creation of information and access services will benefit from the already consistent use and definition of data across systems.
Standard and interoperable communication between services in an SOA is one of the defining features of SOA itself. In the last decade, XML has become a pervasive means for achieving that interoperability, widely used in both the representation of business data and protocol related data. In most SOAs it is likely there will be a requirement for common XML processing functionality such as validation, conversion and enforcement of Web Service related standards. XML Networking technologies can be used to increase message throughputs between SOA services and applications that communicate with one another, and also improve server utilisation through acceleration technologies.
Services in an SOA can themselves be likened to services provided by utility companies in the real world. Individual services are consumed by multiple consumers – applications or other SOA services – accessing them through agreed service level agreements. When a service is first deployed, with future reuse, it is unlikely that all future consumers will be known and there may be troughs and peaks in demand for its functionality. Utility computing solutions such as virtualisation make the relationship between deployed software services and the hardware they are deployed on more fluid. This allows greater flexibility in the way services and applications are deployed in the data centre allowing them to adapt to changing consumer needs more flexibly. Virtualisation technology is a key component of BEA SOA 360º, BEA’s unified SOA platform.
How do companies utilize SOA to best streamline, integrate and manage applications?
Stan Alexander: The theory behind SOA is that the modularisation of your apps into services will yield very efficient, streamlined applications. The premise is that integration is easy through services standards and the process flow is always easily optimised to the business. However, all of this can add complexity. It should move development and integration from custom efforts into more of a streamlined, assembly-oriented task. But now that you have a large number of shared services living on the network, understanding dependencies during upgrades, the impact of new or removed functionality, managing the service level agreements etc. has all become more complicated. The optimal approach is to take advantage of the modularisation, orchestration and standards-based approach, but couple it with a strong services management and governance structure. The streamlining and integration won’t happen by accident, you’ll have to plan for it.
Donald Mowbray: We have worked with a number of early adopter SOA customers – more than 100 organisations – and conducted more than 120 hours of in-depth interviews on how customers are using SOA. The results show that customers have managed to streamline their businesses and applications in three broad areas which we term SOA Value Patterns, one key one being Service-Oriented Integration.
Customers are moving away from traditional Enterprise Application Integration (EAI) technologies and moving towards standards-based integration tools which implement SOA orchestration using BPEL. These benefits have been apparent immediately as the projects have typically involved very rapid change cycles. Furthermore, IT resources with little or no experience were comfortable at maintaining the BPEL processes after a relatively short training period. One customer in the US, a higher education college, delivered their integration project in five days using BPEL and after only three days’ training and five days’ knowledge transfer, two employees with no prior knowledge of BPEL were able to maintain the BPEL-based integration of 22 core processes.
The management of services is absolutely critical to SOA both in terms of access to services exposed outside of the organisation where customers benefit from a Web Services Management solution, that is, who has access to my services? Are they authenticated to do so? For how long are they using my service? We have a number of customer examples where an application has been service-enabled and then exposed outside of that organisation on the web. The need for a management and monitoring solution, such as Oracle Web Services Manager, has been highly beneficial for these customers.
Tom Bishop: Most companies have a heterogeneous mix of different application types built at different times in different languages using different design principles running on different platforms accessed in different ways (batch, client/server, web, etcetera). Regardless of this heterogeneity, all applications are basically containers of knowledge about how to perform certain high-level or low-level tasks, bundled up and exposed for easy use by a non-expert. As an abstraction layer technology, SOA/WS is particularly well-suited to act as the ‘common broker’ or ‘portal’ to all these different types of applications, thus dramatically simplifying use/integration/management of applications.
Malhar Kamdar: Portfolio management of an SOA roadmap rises to the top as the single most important best practice for executing an SOA strategy. Companies that have a future application road map – and, more importantly, an understanding of the future business strategy upon which the application road map is based – have the strongest foundations for an ongoing stream of SOA investment.
Embarking on an SOA initiative is a large undertaking. At the start of most SOA programs, the enterprise is a heterogeneous environment of technical infrastructure, legacy applications, and application development projects. An initial audit of existing applications and projects will identify functionality present in existing systems that should be represented as services.
A mature SOA infrastructure will encompass many enterprise-wide applications. Many companies that pursue enterprise SOA frequently suffer from a lack of consistency in the ways that they create, version, and deploy services. This inconsistency compromises the effectiveness of the SOA strategy, preventing users from easily discovering or reusing shared services and creating inefficiencies in the way that hardware and applications are utilised. In the worst cases, the efficiency of the SOA is reduced, manageability suffers, and service sprawl can become a problem.
To avoid these problems, SOA architects need to create a consistent Enterprise Service Engineering Framework. This maximises the benefits of SOA by providing a proven methodology for delivering projects and services, and helps teams standardise their processes beyond what is offered by traditional software engineering methodologies. This provides the procedures needed to effectively manage and maintain control over all enterprise SOA assets. Key components include common software engineering policies, procedures, and patterns; common engineering disciplines; and common service specifications.
Maria Pardee:This has to be done in the context of an overall architecture and companies have to have a detailed knowledge of the applications that they have and the services that they currently provide, because in many cases several applications will hold the same or similar data and have similar functions. A decision needs to be taken as to where the service should be offered from (which application) and how that is co-ordinated with the other applications that hold the same data to keep them in step. This is important since exposing a fragment of an application or data source as a service in the context of a SOA may effect the overall integrity of the application providing the service.
What are the business benefits of implementing SOA in your enterprise? What are the risks?
Stan Alexander: The real keys are agility and ability to align IT to business. You should also receive some cost savings due to reuse, quicker development cycles, etcetera, but it is through business agility and alignment that large impacts to the business’ margins and revenues occur. Your risk is successfully tracking the development technology, but not the management and organisation changes necessary. Having hundreds of low quality, resource hogging services showing up on your network would be a terrible thing. At EDS, we plan for the on-going life of the SOA solution from the beginning. Tools are installed to manage the service level agreements and to track the life cycle of the services. We measure the organisation against a maturity model that helps everyone to understand where they are and how they impact the business. When the behavioral, organisational and technological changes occur together, then the rewards increase dramatically.
Malhar Kamdar: The greatest impact of an SOA program will typically be in business areas. Given that the average IT budget comprises only five to 11 per cent of the enterprise budget, the gains available from cost containment and higher productivity pale in comparison to the opportunities available from increased business payback.
Continual process improvement and business-oriented delivery of functionality through services is made possible by increased collaboration between business and IT stakeholders. Benefits to keep in mind include how IT accountability to the business strategy is improved and how the cost and benefit of functionality can be tracked on a service-by-service basis.
In our clients’ experiences, the key benefits they have realised from an SOA adoption are:
In contrast the biggest risks that have manifested themselves are:
- Customer service improvement
- Faster time to market
- Information visibility
- New products and services
BEA’s experience with clients has highlighted a number of best practices to mitigate these risks, and realise the benefits. However, the overriding success is by ensuring that the correct sponsor is in place for your SOA implementation. The need for executive level sponsorship is even more apparent when examining SOA budgets. Recent research commissioned by GCR found that over half of respondents had spent at least US$1million on SOA in the last two years, and an estimated 40 per cent of adopters were projecting spend increases of more than US$1million this year. Budgets of this size and projected duration require senior authorisation. While nearly a third of all sponsors were CIOs, other mentions included CFOs and IT directors.
- Timeframe for benefit realisation too long
- Cost escalation
- Getting lost in SOA because of organisation interdependencies
Donald Mowbray: There are clearly many benefits to Service enabling the enterprise:
Risk arises when organisations end up doing ‘SOA by accident.’ What we mean by this term is that customers begin service-enabling their applications with little thought to the overall SOA strategy and how IT is aligned to the business. If IT is purely reactive to the business and has no SOA strategy or roadmap, then organisations will end up effectively creating siloes of SOA.
- Protection of investment, by enabling legacy systems to become part of a modern infrastructure
- Increased Competitiveness - by creating an IT infrastructure that enables an organisation to respond more rapidly to changing environemnts and requirements
- Efficiency - reuse of IT assets (software components within a process can be used elsewhere)
Maria Pardee: As you implement a SOA within an organisation the benefit is that you simplify and abstract your applications to make it easier or faster (cheaper) to change the products or services you sell. You are applying good design practices across your entire systems estate, by applying techniques of pattern matching and abstraction to develop your key business services. The risk is that you make the services too granular (small) and the integration cost (data IO, process orchestration of the services themselves) becomes such an overhead that you lose the benefits of the flexibility that you were trying to achieve. If the business is not ready for change or in need of the flexibility that a SOA offers (i.e. it has steady, non changing processes, or a very static product base) you will end up making your SOA an IT initiative (techies doing IT for the sake of IT) and it will fail. It has to be introduced at a time of business change and be seen to facilitate that change otherwise the business will rapidly lose interest and patience. As you build out the services you have to refine (streamline/automate) processes (get rid of people) and retire systems to help pay for the transformation.
Tom Bishop: The clear benefit of SOA in an enterprise is to reduce the cost/complexity associated with heterogeneity and to replace it with the simplification (across the board) of a more standard way of building, integrating, and managing applications. This simplification reduces cost and increases agility, both of which are a direct benefit to the business. The risks are the security, robustness, and scalability of the technology used to achieve this simplification. When so much depends on it, it had better work.
How can companies bridge the gap between process and resource requirements, when implementing next gen applications?
Stan Alexander: Many next generation applications will look quite different from the traditional application. For one, they may be an assembly or composite of existing functionality. Although some aspects of this have been available for years via portal servers, an enterprise SOA approach is a much better solution. A composite solution can be assembled from services (either your own or vendor provided) in a very clean, standard and efficient manner with SOA.
Another change you’ll see related to composite applications is that the major business processes will no longer be embedded in the application code, but will be visible to business analyst and process specialists who will be able to tailor the process flows with far more ease than traditionally. The impact to the business is that the application processes match real life needs far more quickly and accurately than ever before. This responsiveness should directly lead to efficiencies, cost reductions and the ability to capture business opportunities that otherwise would have been missed.
Maria Pardee: It is useful to get your design teams (in SOA it is better if they are combined systems and process teams) to think in terms of real time processes, that customers run themselves through portals and IVRs (if possible for the products and services you are offering) that are not touched by any employee in you organisation for the duration of the process (i.e. fully automated).
Malhar Kamdar: Companies are thinking not only about technology but also about the organisational structure and processes to support SOA for the long haul. Enterprises often develop a central architecture team or SOA centre of excellence (COE) to help establish policies, mentor and assist distributed development teams, monitor and manage centralised registries, research business requirements, and prioritise services. In some cases, these COEs also create, own, and maintain foundational services and infrastructure for the enterprise.
Even so, many enterprises still rely on a significant amount of manual intervention to capture or enforce protocols, leaving tremendous exposure to error and risk. More strategic moves are warranted, especially when addressing enterprise-scale initiatives and navigating political arenas. When a company starts to scale SOA enterprise-wide the following questions become critical: How do you transform the IT team for SOA? What skills are needed?
Developing skills for SOA is not strictly a linear process, like retraining a COBOL programmer to use Java. New technical skills must be learned, but IT professionals also must develop business acumen that will help them serve their users. Successful re-skilling shifts the view of technical personnel from strictly functional considerations to cross-enterprise concerns. For example, an enterprise architect who originally might have focused on infrastructure details may now need to help business users understand how and why using IT services can benefit the organisation.
CIOs need to consider the following best practices: deploy new or redefined roles, establish core standards, accelerate re-skilling with a plan and use external help judiciously.
Donald Mowbray: In a service-oriented world you cannot always accurately predict the resource requirements of your underlying systems as new processes are created and orchestrated to meet changing business requirements. For this reason, you need a dynamic infrastructure for your SOA that can scale on demand; this is where the notion of grid computing dovetails with SOA. Some companies have particularly demanding application environments, such as a query-intensive portfolio management application in a large financial services company, or a rule-based, price-optimising reservation engine for a global hospitality chain. The ability to dynamically scale the infrastructure on demand is critical to these customers in order to meet peak loads. This can be achieved with an array of grid-based technologies from Oracle, a portfolio which now includes a recently acquired technology for performing in-memory extreme transaction processing in the middle tier.
How does SOA unlock frozen assets and align business with IT?
Stan Alexander: The first step is to examine your application portfolio and identify the assets that could become reusable services. This inventory will likely show redundant assets in multiple locations. However, the true ‘unlocking’ comes when you are able to discover and share these services across a network and tie them into a business process. This coupling of BPM with SOA allows the value of that software asset to be shared across multiple silos. And when IT and business begin sharing a common understanding of the business process, then they begin to become truly aligned.
Malhar Kamdar: SOA has the promise of delivering:
However, just by building out services that can be reused does not necessarily deliver business agility. Tighter alignment between business and IT is needed to ensure that IT agility (delivered through a service-oriented architectural approach) can translate into true business agility. BEA believes that there is a next level of business agility that leverages SOA but builds on it. Business processes, by their very nature these days, cross organisational and system boundaries. Business velocity, and the next level of business agility, can be achieved through the focused design, implementation, execution, and optimisation of these business processes. By abstracting business process logic out of the applications (where it’s traditionally been hard coded and difficult/costly to change) and leveraging the core principles of a service-oriented architecture, organisations can truly realise the next level of business agility through process-centric composite applications. Additionally, with business process management, business and IT can more closely collaborate to ensure that what the business requires is what is delivered at the end of the day. This now translates IT agility into true business agility.
- Applications are componentised into business services
- ervices can be combined and reused more quickly in the assembly of new and loosely coupled business processes and composite applications
- The promise is there to drive positive ROI by reducing costs and increasing agility over time
Donald Mowbray: A lot of organisations have a whole raft of legacy systems, platforms, applications and code – that are essentially frozen assets because the application functionality is locked into a monolithic mainframe, for example. SOA enablement is an easy way to begin to address IT modernisation by wrapping existing application interfaces using SOA wrappers and creating SOA services on top of the existing legacy infrastructure. This is completely in line with the SOA mantra of not ‘ripping and replacing.’ In fact, ripping out mainframe applications is often a hugely cost prohibitive thing to do.
In terms of the approach, we have a number of examples. One, a large insurance company, faced huge financial drains with its 35-year-old mainframe application. The key problem was that any bugs that resulted from a minor change to the COBOL application would involve 30 engineers, all experts on a particular part of the application, sitting in a room to work out what might have happened. This was then followed by weeks of diagnostics to resolve the issue.
This company decided to use SOA techniques to retire the mainframe over time. First, they exposed the business logic in the mainframe application as services and then re-built the process layer in BPEL on top of this services layer. The benefits were immediate. Issue resolution times were cut as they could identify problems related to specific services. They were also able to improve their diagnostics by layering a web services management infrastructure that monitored the services layer.
Tom Bishop: SOA unlocks frozen assets in two ways. First, it makes available resources (in the form of applications or parts of applications) to users who might not have been able to get at them before. It also makes applications more dynamic, in that the traditional link between an application and its underlying platform is now loosened with SOA. This makes it easier to more closely align available underlying computing resources to where they are needed most. This also answers the first part of the alignment question.
The second part of the alignment question is answered by thinking about the agility that results from being able to quickly combine, uncombine, and recombine low level applications and application components via SOA into new applications in much less time. This makes it far easier in theory to help IT align with changing business needs.
Maria Pardee: Properly implemented and executed, an SOA requires very close working of the IT department with the business process designers and the business operations teams as it will eventually touch all aspects of the organisation. Correctly deployed it will remove duplication from the organisation, both within the IT estate and across processes as there will only be one way of performing a particular task. This will free up customer service agents to address higher value activities, enable the closure of duplicate IT systems and enable wide scale process redesign across the enterprise making it more efficient.
Why is an effective internal approach to SOA organisation, and governance key to its success within an enterprise?
Stan Alexander: The breakdown of silos will force sharing and cooperation where minimal had happened in the past. How do you make decisions? What if different units have differing SLAs? Who pays for a shared service? Your organisation and culture will have to adapt to this new model. Add in the additional complexities of managing hundreds of services, securing them, versioning them, etcetera and you can see why a strong governance model to provide vision, standards and decision making is key.
Maria Pardee: By definition SOA requires that things are done once and only once within an enterprise. This requires discipline or effective governance across the enterprise, something that is consistently mentioned as a problem when I talk to other people involved in SOA implementations within large organisations.
Tom Bishop: When so much is riding on a technology like SOA, to deliver all the benefits outlined above, having an effective internal approach to SOA organisation and governance is mandatory. Without this, SOA simply translates one form of complexity (hence cost) into another, and there is no net benefit for the time and effort expended.
Donald Mowbray: As I said earlier, we don’t want customers to do SOA by accident. Effective governance policies and procedures need to be put in place to ensure that SOA is ‘SOA by design’ and this is where governance comes in. Unfortunately, governance is an often-misunderstood term in relation to SOA. Some people use SOA governance to mean service lifecycle governance, meaning governing the lifecycle of services from creation through deployment. Others take it to mean applying runtime policies to services. In fact it means much more than this in our opinion. SOA governance is about making sure you deliver on your SOA and business goals. To do this, you need a plan that we refer to as the SOA Roadmap. The SOA Roadmap is simply an outline of the capabilities and policies that need to be put into place over a period of time (such as two-five years) to ensure that you deliver on your business and SOA strategy. By incrementally building the required capabilities over a period of time, you can increase your SOA maturity, thereby enabling you to deliver more projects in a more efficient and change-resilient way.
Malhar Kamdar: It’s not enough to address the technical challenges for deploying a service-oriented architecture. Any initiative that aims to transform the delivery of IT to the business can succeed only by adopting a genuinely strategic, enterprise-wide, approach to the problem by establishing a best practice organisation and governance structure from the outset. SOA is no exception.
Governance and developing an organisation to support governance is the foundation of successful SOA implementation, and a strong SOA governance program will control all of the following areas. In fact, organisation and governance can be the biggest determinant of SOA success, and must embrace the following key program areas, which are best practices that BEA has seen with many clients:
- Prioritise the SOA benefits: Force the full IT leadership team to prioritise the benefits the enterprise and the divisions will realise by adopting an SOA approach
- Establish the boundaries: Debated and agree at the top on foundational organisation and architecture guidelines, such as ‘data is owned by the Enterprise’
- Develop a conceptual SOA organisation: Create and privately syndicate ‘Straw’ organisational models as a proven effective way to surface the issues among the leadership team with minimal territorial behaviour
- Address core processes first: Not all software development processes need to be addressed immediately, and in fact addressing too much too early can be detrimental to success. Successful organisations have begun to address four core processes: SOA architectural management, SOA project management, SOA alignment, and SOA infrastructure management
- Ease culture shock: An effective approach to rollout is to let the number of services actually deployed in production set the pace, allowing the organisation and governance changes to be implemented in phases
- Oversight committee: Make establishing an oversight/review committee with a priority, not an afterthought
How do enterprises overcome challenges associated with service lifecycle, technology standards and team roles when migrating to an SOA environment?
Stan Alexander: This is an excellent question that is often overlooked. A service does go through a life cycle of being published, upgraded (versioned) and eventually retired; these services will have dependencies on other services that must be managed. Your technology platform will also go through upgrades and standards will evolve while new ones emerge. Part of the answer is that you need to plan ahead and have the proper tools to support the life-cycle management. But even more so, this points to the need for a good architectural design to address these issues. Proper use of server and service virtualisation, advertising the services correctly, etcetera will give you the flexibility to manage this scenario.
Malhar Kamdar: The key to successful SOA migration is planning! From the outset, one of the core disciplines of SOA is to establish a reference architecture and standards designed to outline standards for service definition, message formats, naming conventions, security policies and other protocols. This often includes guidelines on specific architectural and service design patterns and the use of technologies to uphold the reference architecture, which must be the foundation of your SOA program.
This must be combined with effective service life-cycle management to define processes and policies, and ensure all service life-cycle steps are properly addressed, from requirements gathering through to service retirement. This requires guidelines and standards for creating and publishing services to versioning and change management procedures and controls.
Re-skilling for enterprise SOA requires organisations to deploy new roles and redefine existing roles within the IT organisation. Redefined roles are required when functions change in scope and responsibility to meet new business requirements. However, this change must be executed incrementally, minimising disruption and resistance, and ensuring team continuity whilst evolving the necessary change.
New roles include an IT executive, who acts as the ‘mayor’ and leads the SOA charge; the enterprise architect, a ‘city planner’ who drives the technical and design requirements; a service architect, the ‘building code engineer’ who champions architectural consistency throughout service design; service engineers, the ‘building contractors’ who oversee the creation and assembly of services; and the developers and administrators, the ‘builders’ who develop and maintain service-based applications.
Donald Mowbray: As I described earlier, governance can be defined as the interaction between policies (what), decision-makers (who), and processes (how). Companies that mature their SOA have established successful governance solutions to help individuals make the proper decisions within the context of the problem space. These companies have also achieved an effective layering of SOA capabilities in areas such as technology, standards, architecture, governance, organisational structure, processes, and strategy. SOA and SOA governance require a change of culture, especially within companies that have legacy systems and siloed operations. An SOA roadmap built using a maturity model, such as our Five-Level SOA Maturity Model, allows companies to begin the SOA journey, leveraging and building on each successful step.
Most large companies that have been developing technical solutions for years either internally or through acquisitions, find that they end up with redundant teams that rely on different vendor products or home-grown solutions, yet provide the same basic functionality. Companies must strive for simplicity in their technology infrastructures, and avoid vendor lock-in by selecting vendor solutions that are based on industry standards and equally important-choosing vendors that are active members of standards organisations, such as W3C, OASIS, and WS-I. Companies must focus on their requirements and weight them before moving forward to select a vendor tool.
As companies start their SOA journey, typically a small group of architects and developers drives solutions. Since SOA is essentially a philosophy within enterprise architecture, it is important that the EA and/or Integrated Competency Centre (ICC) groups establish training. Training also must be provided for the individuals supporting the SOA tools, as well as the project managers (PMs), business analysts (BAs), quality assurance (QA) team, and others who interact with the design, implementation, deployment and maintenance of the SOA solutions.
Tom Bishop: As with any transformation of this importance, scope and magnitude, one needs only to fall back on the tried-and-true lessons of ‘people, process, and technology.’ In this regard, the technology is actually the easiest to solve, as there are quite a few products/solutions on the market that do a good job of addressing the technical requirements. People and process are often overlooked as key elements of a successful migration, and thus the reason why most of these migrations fail to deliver the business benefits intended. So, the recommendation is to pay particular attention to these two elements through education, communication, and project management discipline.
Maria Pardee: As enterprises migrate to an SOA environment different skills will be valued in the organisation. The technical guru who knows how application A works in detail is no longer as valuable as the person who can liberate the functionality of application A as a set of services and maybe even enable application A to be switched off, as other applications provide those services. Version control of the services is essential; hence you will need a repository or tool to hold the services that make them freely available to the other interested parties in the enterprise (for example designers/developers need access to the services during the implementation phase of new projects to ensure that re-use is maximised).
Standard processes need to be defined so that the IT services can be developed to meet the needs of the enterprise, and the flexibility points of the process identified so that services can be built at the correct level of granularity. As a SOA is being built within an enterprise it will be more successful if the enterprise is itself service orientated in its approach to product and service design and delivery. Emerging technology standards have made the implementation of SOA easier, but a common data exchange model and a standard integration technique (enterprise service bus or some such technique) needs to be enforced to keep the sevices that are being developed consistent with each other.
How can SOA eliminate the current “siloed” IT and business functions within enterprises?
Stan Alexander: As discussed earlier, SOA does not do this by itself. What SOA gives you is a powerful approach for improving the agility of your enterprise. But to truly reap the rewards, you must couple it with a culture of reuse, change the organisation to allow cross-silo collaboration, install a meaningful governance process and couple SOA technology with BPM to really drive the business value.
Maria Pardee: By the adoption of standard service orientated business processes and the development of a set of services from the IT applications that can be integrated to deliver the standard business processes means, organisations have the chance of doing things once and in a consistent manner independent of specific products or services which will help to eliminate silos.
Malhar Kamdar: The gap between business need and IT execution often has less to do with the technology leveraged than the way in which IT as a whole is delivered to the business. Often companies develop a business strategy and then attempt to implement it through a separate IT strategy. The result is a siloed IT program that does not support the enterprise as a whole.
The challenge in achieving productivity gains is to close the gap between business need and IT execution. This requires fundamental changes to the alignment of IT and business strategies. In implementing our first generation SOA, BEA found that it was indeed the alignment of IT and business strategy through an SOA programme that closed the gap between business need and IT execution.
Tom Bishop: Again, as noted above in several earlier answers, SOA reduces or removes the traditional ‘friction’ in how, where, and why certain types of application functionality get used. Used in the correct way, this technological ‘oil’can help reduce the barriers between parts of the business or even part of the IT organisation (by increasing code reuse, data sharing, and process integration).
Donald Mowbray: As I’ve mentioned already, existing applications can be wrappered as services and exposed outside of traditional, monolithic or ‘siloed’ applications. The application of standards-based workflow languages, such as BPEL, allows these services to be orchestrated into new business processes that match the rapidly changing business requirements of a particular organisation. At Oracle, we’re taking this approach a step further by using SOA as the key underlying principle to develop our next generation of enterprise applications, which we call Fusion Applications. These applications are being engineered, from the ground up, on the principles of SOA and using the standards-based technologies afforded by Oracle Fusion Middleware. In the interim, the infrastructure supporting this next generation of applications is being shipped as a set of pre-integrated process flows operating across traditional application siloes. Oracle Application Integration Architecture provides customers with pre-built integrations and ongoing support and enhancements on these integrations.
Is it best to leave it to the experts when it comes to SOA integration for your enterprise? How can SI/outsourcing partners streamline this process?
Stan Alexander: Only you truly know your own enterprise and its business, but there are a number of advantages to partnering with a firm like EDS. The broader view of how other firms, inside and outside of your industry, have solved similar problems can prove invaluable to your design, experience with a broad array of technology and the ability to leverage resources globally can improve your efficiency and effectiveness. When speaking of the outsourcing model, the ability to mitigate your risk with joint service level agreements and the freedom it gives you to focus on your core business could make or break your business case.
Donald Mowbray: The customer example I cited earlier proves that the skills required for service oriented integration can be picked up fairly rapidly, especially when one compares the level of training and expertise required for traditional integration broker technologies. However, effective implementation and deployment of SOA at an enterprise level will involve an array of skilled resources and companies should look at identifying a key SOA platform provider and a key implementation partner to help them reach greater levels of SOA maturity. Customers should also look to organisations who have a set of tools, best practices and methodologies for building a SOA roadmap.
Maria Pardee: SOA integration needs to be done in a consistent and efficient way if your SOA is going to scale and perform. There are a limited number of well documented integration techniques that are associated with SOA (for example, web services via some form of enterprise service bus). However you have to look at the nature of the information exchange that you are trying to achieve between each service in your SOA and develop a set of rules for the different data exchange models. Depending on the size/experience of your IT department then this may be something that you have the knowledge to manage in-house, otherwise there are many SIs and integration tools vendors who can help in this area.
SI/outsourcing partners can probably streamline this process as they are likely to have already built an information exchange capability for a previous client so they should have the necessary experience to streamline the process for an organisation without the people or experience.
Tom Bishop: I think the best way to answer this question is to first think about what the organisation is trying to accomplish with SOA (objectives/requirements), then honestly assess where the organisation is currently regarding knowledge/skills, and where the organisation either has or believe it needs to have that knowledge and/or skills in-house as a core competence going forward. Where it’s necessary to keep those in-house, it is advisable to start acquiring that knowledge and those skills through all the traditional means (education/training/hiring). Then out-source all the rest. SIs/outsources can help in both regards (helping define requirements and doing the assessment, then acting as the external agent where some outsourcing is indicated).
Malhar Kandar: How CIOs deploy personnel resources depends on the organisation’s global sourcing strategy. Yet successful deployment can be assured only when core standards and governance are in place. These standards must include defined job skills requirements that are incorporated into company job descriptions and incentive structures.
With these standards in place, CIOs are prepared to make specific global sourcing decisions. Insourcing such elements as SOA architecture and governance services are best practices that help SOA run smoothly. In addition, certain design services and critical services builds are most effective when done in-house.
Hiring professional service consultants, process experts or outsourcing the entire development centre can, in the right circumstances, effectively help a company get immediate skill set help and drive necessary change, while minimising disruption. It is critical, however, to ensure that this external knowledge is transferred to internal staff. Properly planned and used, external expertise can serve as an excellent source for on-the-job training, mentoring, and skills transfer.