|By Michael Poulin||
|January 8, 2007 01:45 PM EST||
When approaching a SOA implementation, I would like to consider two fundamental questions that many developers ask:
1) What's the difference between service-oriented and service-based architectures?
2) What special architecture elements are defined by the SOA RM?
In my opinion, the answer to the first is in the difference between the words oriented and based. I believe that smart IT organizations offer a lot of services already because the technical benefits of services have been well known for a while. However, the applications based on these services are still monolithic and don't provide convenient ways to implement business change. SOA, by contrast, elevates services to the level of first-class citizens by associating them (well, some of them) with business responsibilities. That is, business tasks can finally surface in the form of aggregated services that are open to business changes by design. SOA RM stresses a new role and meaning for services and outlines that:
1) "The central focus of Service Oriented Architecture is the task or business function" versus both IT task/function and business operation automation task. This means that SOA primarily targets actual business services and processes instead of just improving existing (obsolete or legacy) applications.
2) SOA service may be MUCH richer than a technical service because SOA service is supposed to run in the "execution context." The latter is defined as the set of technical and business elements that form a path between those with needs and those with capabilities. The "business elements" open absolutely new sphere of conditions, particularly business conditions that a regular technical service doesn't deal with.
3) SOA "reflects business-motivating considerations" expressed "in service descriptions and service interfaces", where the service description can be expressed in the form of a contract. The service contract includes policies that "MAY also express business-oriented policies - such as hours of business, return policies, and so on".
If you elaborate on the business aspect of SOA, you'll get a much different picture of the service realm with different purposes, use, governance, maintenance, and, finally, architectural concept than we have in a "regular" service-based architecture. Besides these, I don't see any real differences. The service serves; the questions are how, where, why, when, to whom, and if it's needed after all.
The second question is trickier. The SOA RM standard doesn't articulate much on architecture elements. Instead, it defines SOA services and their relationship in new way that allows for addressing actual business services and processes by assigning business responsibilities to the services directly. That is, the standard defines what an IT service has to be oriented to in order to provide for IT agility to the business. I understand that the intention of SOA RM is giving us the definitions of the service and the "associates" that require a business service-oriented infrastructure. This infrastructure is the very architecture meant by the reference model.
So, since we're going to discuss SOA design pillars, let's list some commonly used elements of SOA infrastructure that include:
- a SOA Service Communication Framework
- a SOA Service Composition Framework
- a SOA Service Transaction Framework
- a SOA Service Security Framework
- User Interfaces (including human UI) and
- a SOA Service Component Adapter Framework
Going forward, let's notice that some pillars relate to the SOA service design while others relate to the service interactions with other services and its consumers. Pillars 1 to 5 have ordered dependencies, the rest are applicable to the entire design process. Plus we'll use terms that have been defined and discussed in part one of this article, so you might look through it.
Service Design Pillars
Considering Business Services selected for the implementation in SOA identify technical "vertical" and "horizontal" services. A vertical service implements one or a few business tasks while a horizontal service implements a supporting SOA infrastructure task, e.g. a Service Repository or Service contract management function. Traditional technical services like security, login, and the like also fit into the horizontal services category.
While the principle of separation of concerns is fully applicable here, the tasks implemented by the same vertical service have to provide "cohesive collections of behaviors" from the business perspective. Plus technical services have to be reasonably coarse-grain in their interfaces (to minimize the network round-trips and future modernizations in production caused by new technologies) but still provide flexibility. Nevertheless, one has to expect a certain amount of exploration and refactoring in this area of the service lifecycle.
When identifying horizontal services, first consider scalability, reliability, performance, expendability, and manageability - all together; consider programming convenience second. Services that are too generic usually don't provide the best performance just as too detailed ones cause problems in aggregation, management, and maintenance in light of frequent changes in the execution context.
Considering business units of work, identify business data and its flows. Take into account two orthogonal sets of requirements - the presentation layer and the persistence layer requirements. Some units of work can require compositions of tem, so watch for a clear information transition with layered data format mappings. Also consider combinations of the business and just the technical data in the flows. That is, define metadata for the business and technical data upfront and operate at the metadata level (UML and WS-CDL can help).
Metadata is an important SOA element. Metadata provides the necessary data abstraction and allows for a data-quality controls. An XML Schema is one possible metadata definition and control instrument. This doesn't mean that only XML format is recommended. In some cases, especially for synchronous short-running transactions, programmatic Data Objects may be used for performance improvement. However, the Data Objects better be derived or generated from the XML Schema, which would provide metadata enforcement (related utilities are available in Java and C# and are known as XML binding).
Define and lock down service interfaces and interfaces to the service resources such as persistence storage. Design the interface as business-oriented coarse-grained so your consumers can avoid frequent refactoring.
Best practices recommend avoiding RPC-like style in service interfaces. Web Services, RPC encoding, and SOAP-encoded arrays, in particular, seriously restrict service interoperability; consider a document/literal style instead. For non-XML cases, use a container style of interface to pass data structures and "exchanged" operations (aka commands in the Command Pattern).
It's important to design service interfaces with an eye to security, transaction, manageability, error handling, versioning, and interoperability from the beginning. Adding any of these things later is either extremely expensive (in effort, time, and money) or practically impossible.
For example, security is very important these days because of multiple technology threats and industrial/corporate security policies. Transaction support is dictated by its potential participation in service compositions. Manageability and error handling are core to any business service and process. Interoperability is one of the most critical factors in the SOA service lifecycle. That is, use of specific programming platform constructs can significantly reduce the utility of a service and lead to its refusal.
In spite of any ideas about keeping the service interface immutable, in real life variations in service invocation always happen over time. Anyway, a service interface isn't the only one to constitute a SOA service; there are also possibilities of SOA service behavior changes that affect service consumers. That is, the service has to be versioned. The versioning may be expressed via the interface and/or provided by a service registry. Only one condition has to be preserved in this regard - no service parts including the interface may be versioned separately from the service itself for service consumers.
A service access protocol may be considered a part of the service interface definition or a separate issue. The spectrum of communication protocols used in SOA is wide including TCP/IP, HTTP/HTTPS, SMTP, and IIOP.
The designer has to choose communication protocol(s) for every SOA service. When considering a protocol, one has to take into account: 1) invocation model - synchronous or asynchronous; 2) the amount of transferred data; 3) performance; 4) extendibility; 5) scalability; 6) reliability; 7) transaction; and 8) manageability.
Protocols are also important for communicating with service resources. It's better to use standardized protocols where possible. For example, current best practice refers to the Web Services Invocation Framework (WSIF) as a binding bridge between Web Service's WSDL and a particular service resource protocol while Java components rely on JBI.
The right protocol is a balanced choice. For example, HTTP/SOAP-based protocol is slower than IIOP but it's more extendible; TCP/IP is more reliable than UDP but it's slower. Don't forget that one service may have more than one communication channel. For example, the same session EJB can be reached through RMI-over-IIOP and SOAP-over-HTTP/S.
The pioneers of SOA implementations have recommend doing a "protocol performance POC" for particular kinds of tasks. That is, the earlier in the design process a performance test gives the results the better for the entire project because refactoring in this area means costly redesign from the scratch.
Define the SOA Service contract and Execution Context. One of the most important parts of the SOA service design is the definition of the SOA service contract template. While SOA RM simply says that "a contract...represents an agreement by two or more parties. Like policies, agreements are also about the conditions of using a service; they may constrain the expected real-world effects of using a service," the SOA service contract may be relatively complex. The contract, preferably written in XML, defines among other things:
- runtime policies/constraints
- runtime functional service parameters exposed to the service consumers; these parameters can be monitored and controlled
- non-functional service parameters like scheduled service availability
- business functional characteristics promised to service consumers like gathering audit information on service invocation events (the characteristics should be evidential and measurable)
- all service interfaces and communication protocols exposed to a particular consumer
- definitions or references to the Change Control procedures related to the service
- service version information
- references to the external (to your organization) SOA services invoked by the service. It's important for the business to know what other SOA services are involved because some political and business considerations may cause certain constraints on such involvement
- some custom or business domain specific parameters if needed
Runtime policies better be derived from the SOA service contract. The policies have to follow standards wherever possible. As SOA RM defines it, "A service contract is a measurable assertion that governs the requirements and expectations of two or more parties. Unlike policy enforcement, which is usually the responsibility of the policy owner, contract enforcement may involve resolving disputes between the parties to the contract. The resolution of such disputes may involve appeals to higher authorities."
The SOA service implementation process treats the service contract as an objective set of development tasks. Not all implementation tasks ought to be expressed in the service contract, but only those that get visibility to the service consumers. Other tasks are the subject of regular application requirements.
Another fundamental characteristic of a SOA service is the execution context. According to SOA RM, the "Execution context is central to many aspects of a service interaction." The context includes concepts such as the SOA service contract, and behavioural and information models. The same service can be viewed and act differently in different execution contexts. For example, a logging service may store data "in the clear" when serving load-balancing calculations but it has to encrypt data for financial risk calculations (i.e., financial regulations constitute the execution context).
Develop for security. SOA security isn't included in the SOA RM. Nonetheless it's obvious that the model's "real-world effects" can't survive without a business trust, i.e., without service security. We don't recommend developing a special implementation for each service. Instead use corporate security services at the level of service communication and service components, e.g., security policy syndication services.
Since a lot of security threats come from inside an organization, perimeter protection (external firewalls) isn't enough. We can outline the minimal SOA security requirements as 1) end-user bi-directional authentication with the service, and 2) inter-service bi-directional authentication. End-user and inter-service authorizations are much desired but may be optional in particular cases (e.g., a consumer may access everything after he's confirmed as a legitimate user). Other security measures are optional as well and depend on the nature of business data and processes.
Traditional security means like PKI, Kerberos, SSL, two-way SSL, user name and password tokens serve well in SOA. The service-specific means relate to the XML security standards. They include XML Digital Signature, XML standards for PKI (XKMS), SAML (for propagating the subject and its credentials), XACML (for transferring security policies and policy validation controls), and other encryption techniques applied to the exchange messages and communication channels. The choice of security methods to be applied depends on corporate and industry policies and regulations.
The takeaway here is that whatever security means ought to be implemented, they have to be considered in the design from the beginning. It's almost impossible or extremely expensive to add security in later phases of development.
Develop to support transactions. SOA RM recognizes the importance of service transaction as "higher-order attributes of services' process models" but doesn't detail the topic. In practice, SOA uses commonly accepted standards for transaction management on different platforms and adds cross-platform transaction management standards. Transactions can be used as in an aggregated service (choreographed aggregation) as in an orchestration of services reflecting fragments of Business Process Management (BPM). The orchestration is typically based on the BPEL standard that introduces short-running and long-running transactions.
A short-running transaction is usually a synchronous RPC-type invocation running according to the known rules of transaction isolation. Transaction standards for Web Services and messaging include WS-Transaction and WS Reliable Messaging (WS-RM). A short-running transaction is good for performance and commonly used for co-located services or in cases where failure is unlikely and error handling is simple. A long-running transaction is an asynchronous transaction where the transactional state may be persisted for long periods of time (days, months, years) and are restored after a certain event. This kind of transaction is quite useful in manual operations or human decision making in the automated business process. BPEL also introduces a transaction capable of "undoing" the mistaken results of the previous transaction by compensating them (like an overcharge refund).
Transactional behaviour has to be embedded in the SOA service from the design phase. This will help provide future service aggregation- and orchestration-based SOA solutions that might not be clear at the moment the service is initially designed.
Design for interoperability. As mentioned, interoperability is a crucial and critical characteristic of a SOA. The SOA RM says, "The value of SOA is that it provides a simple scalable paradigm for organizing large networks of systems that require interoperability to realize the value inherent in the individual components."
The recommendation here is to put a lot of attention on the data types and naming convention policies (besides the protocols) on the both sides of the interface - for the requester and the provider components. The policies must enforce compatibility and interoperability. One possible mechanism for data type and naming space interoperability control is XML Schema or any other schemas (metadata) accepted by the requester and the provider.
Another best practice recommendation is the creation of a proxy (isolation mapping layer) around the interfaces. That is, the data formats passed through the service interface shouldn't be driven by internal data formats (for either requester or provider). If the evolution of either side causes data format changes, it's better to create object mapping in the proxy than to try to modify the interface.
Use Rules for flexibility and easy-to-change implementation. Rule engines are just tools used in SOA infrastructure implementation and certainly not a part of the SOA RM. However, being combined with policies on one side and decision-making points on the other side, rule engines become quite powerful instruments for providing a high level of flexibility and quick adaptation to business changes. Leading SOA infrastructure/framework vendors include different rule engines into their business process design tools and Enterprise Service Bus solutions.
While rule engines are really convenient instruments, there's no compatibility between engines from different vendors; a rule engine locks you into a vendor. Anyway, not everything should be driven by rules - rule maintenance is expensive and requires special skills. That is, the designer and developer have to be reasonable in using rule engines.
For example, if the business logic introduces a point for choosing from a finite number of cases (while cases change seldom), using rules may be overkill. Another known risk in using rules is a potential rule contradiction. That is, a rule engine generally doesn't automatically validate if a new rule contradicts any existing rules and any permitted combinations of those rules. Without such validation, the new rule can bypass restrictive rules and expose prohibited information (e.g., a junior clerk gets access to a high-volume account). So when using a rule engine in SOA, proceed with caution.
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
Aug. 25, 2016 01:15 AM EDT Reads: 1,593
The emerging Internet of Everything creates tremendous new opportunities for customer engagement and business model innovation. However, enterprises must overcome a number of critical challenges to bring these new solutions to market. In his session at @ThingsExpo, Michael Martin, CTO/CIO at nfrastructure, outlined these key challenges and recommended approaches for overcoming them to achieve speed and agility in the design, development and implementation of Internet of Everything solutions wi...
Aug. 25, 2016 12:45 AM EDT Reads: 1,898
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Aug. 25, 2016 12:00 AM EDT Reads: 2,956
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
Aug. 24, 2016 09:15 PM EDT Reads: 1,661
SYS-CON Events announced today that Venafi, the Immune System for the Internet™ and the leading provider of Next Generation Trust Protection, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Venafi is the Immune System for the Internet™ that protects the foundation of all cybersecurity – cryptographic keys and digital certificates – so they can’t be misused by bad guys in attacks...
Aug. 24, 2016 04:15 PM EDT Reads: 2,555
Pulzze Systems was happy to participate in such a premier event and thankful to be receiving the winning investment and global network support from G-Startup Worldwide. It is an exciting time for Pulzze to showcase the effectiveness of innovative technologies and enable them to make the world smarter and better. The reputable contest is held to identify promising startups around the globe that are assured to change the world through their innovative products and disruptive technologies. There w...
Aug. 24, 2016 04:15 PM EDT Reads: 352
SYS-CON Events announced today Telecom Reseller has been named “Media Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
Aug. 24, 2016 04:15 PM EDT Reads: 481
Smart Cities are here to stay, but for their promise to be delivered, the data they produce must not be put in new siloes. In his session at @ThingsExpo, Mathias Herberts, Co-founder and CTO of Cityzen Data, will deep dive into best practices that will ensure a successful smart city journey.
Aug. 24, 2016 02:15 PM EDT Reads: 1,438
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Aug. 24, 2016 12:00 PM EDT Reads: 3,850
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Aug. 24, 2016 11:00 AM EDT Reads: 2,113
In today's uber-connected, consumer-centric, cloud-enabled, insights-driven, multi-device, global world, the focus of solutions has shifted from the product that is sold to the person who is buying the product or service. Enterprises have rebranded their business around the consumers of their products. The buyer is the person and the focus is not on the offering. The person is connected through multiple devices, wearables, at home, on the road, and in multiple locations, sometimes simultaneously...
Aug. 24, 2016 09:45 AM EDT Reads: 2,255
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
Aug. 24, 2016 09:00 AM EDT Reads: 3,514
For basic one-to-one voice or video calling solutions, WebRTC has proven to be a very powerful technology. Although WebRTC’s core functionality is to provide secure, real-time p2p media streaming, leveraging native platform features and server-side components brings up new communication capabilities for web and native mobile applications, allowing for advanced multi-user use cases such as video broadcasting, conferencing, and media recording.
Aug. 24, 2016 07:45 AM EDT Reads: 2,090
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Aug. 24, 2016 07:15 AM EDT Reads: 1,661
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Aug. 24, 2016 07:00 AM EDT Reads: 1,759
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
Aug. 24, 2016 04:30 AM EDT Reads: 2,187
Akana has announced the availability of version 8 of its API Management solution. The Akana Platform provides an end-to-end API Management solution for designing, implementing, securing, managing, monitoring, and publishing APIs. It is available as a SaaS platform, on-premises, and as a hybrid deployment. Version 8 introduces a lot of new functionality, all aimed at offering customers the richest API Management capabilities in a way that is easier than ever for API and app developers to use.
Aug. 24, 2016 02:00 AM EDT Reads: 1,410
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abil...
Aug. 24, 2016 01:45 AM EDT Reads: 1,886
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
Aug. 24, 2016 12:00 AM EDT Reads: 2,850
I wanted to gather all of my Internet of Things (IOT) blogs into a single blog (that I could later use with my University of San Francisco (USF) Big Data “MBA” course). However as I started to pull these blogs together, I realized that my IOT discussion lacked a vision; it lacked an end point towards which an organization could drive their IOT envisioning, proof of value, app dev, data engineering and data science efforts. And I think that the IOT end point is really quite simple…
Aug. 23, 2016 11:30 PM EDT Reads: 2,236