|By Michael Poulin||
|January 8, 2007 01:45 PM EST||
When approaching a SOA implementation, I would like to consider two fundamental questions that many developers ask:
1) What's the difference between service-oriented and service-based architectures?
2) What special architecture elements are defined by the SOA RM?
In my opinion, the answer to the first is in the difference between the words oriented and based. I believe that smart IT organizations offer a lot of services already because the technical benefits of services have been well known for a while. However, the applications based on these services are still monolithic and don't provide convenient ways to implement business change. SOA, by contrast, elevates services to the level of first-class citizens by associating them (well, some of them) with business responsibilities. That is, business tasks can finally surface in the form of aggregated services that are open to business changes by design. SOA RM stresses a new role and meaning for services and outlines that:
1) "The central focus of Service Oriented Architecture is the task or business function" versus both IT task/function and business operation automation task. This means that SOA primarily targets actual business services and processes instead of just improving existing (obsolete or legacy) applications.
2) SOA service may be MUCH richer than a technical service because SOA service is supposed to run in the "execution context." The latter is defined as the set of technical and business elements that form a path between those with needs and those with capabilities. The "business elements" open absolutely new sphere of conditions, particularly business conditions that a regular technical service doesn't deal with.
3) SOA "reflects business-motivating considerations" expressed "in service descriptions and service interfaces", where the service description can be expressed in the form of a contract. The service contract includes policies that "MAY also express business-oriented policies - such as hours of business, return policies, and so on".
If you elaborate on the business aspect of SOA, you'll get a much different picture of the service realm with different purposes, use, governance, maintenance, and, finally, architectural concept than we have in a "regular" service-based architecture. Besides these, I don't see any real differences. The service serves; the questions are how, where, why, when, to whom, and if it's needed after all.
The second question is trickier. The SOA RM standard doesn't articulate much on architecture elements. Instead, it defines SOA services and their relationship in new way that allows for addressing actual business services and processes by assigning business responsibilities to the services directly. That is, the standard defines what an IT service has to be oriented to in order to provide for IT agility to the business. I understand that the intention of SOA RM is giving us the definitions of the service and the "associates" that require a business service-oriented infrastructure. This infrastructure is the very architecture meant by the reference model.
So, since we're going to discuss SOA design pillars, let's list some commonly used elements of SOA infrastructure that include:
- a SOA Service Communication Framework
- a SOA Service Composition Framework
- a SOA Service Transaction Framework
- a SOA Service Security Framework
- User Interfaces (including human UI) and
- a SOA Service Component Adapter Framework
Going forward, let's notice that some pillars relate to the SOA service design while others relate to the service interactions with other services and its consumers. Pillars 1 to 5 have ordered dependencies, the rest are applicable to the entire design process. Plus we'll use terms that have been defined and discussed in part one of this article, so you might look through it.
Service Design Pillars
Considering Business Services selected for the implementation in SOA identify technical "vertical" and "horizontal" services. A vertical service implements one or a few business tasks while a horizontal service implements a supporting SOA infrastructure task, e.g. a Service Repository or Service contract management function. Traditional technical services like security, login, and the like also fit into the horizontal services category.
While the principle of separation of concerns is fully applicable here, the tasks implemented by the same vertical service have to provide "cohesive collections of behaviors" from the business perspective. Plus technical services have to be reasonably coarse-grain in their interfaces (to minimize the network round-trips and future modernizations in production caused by new technologies) but still provide flexibility. Nevertheless, one has to expect a certain amount of exploration and refactoring in this area of the service lifecycle.
When identifying horizontal services, first consider scalability, reliability, performance, expendability, and manageability - all together; consider programming convenience second. Services that are too generic usually don't provide the best performance just as too detailed ones cause problems in aggregation, management, and maintenance in light of frequent changes in the execution context.
Considering business units of work, identify business data and its flows. Take into account two orthogonal sets of requirements - the presentation layer and the persistence layer requirements. Some units of work can require compositions of tem, so watch for a clear information transition with layered data format mappings. Also consider combinations of the business and just the technical data in the flows. That is, define metadata for the business and technical data upfront and operate at the metadata level (UML and WS-CDL can help).
Metadata is an important SOA element. Metadata provides the necessary data abstraction and allows for a data-quality controls. An XML Schema is one possible metadata definition and control instrument. This doesn't mean that only XML format is recommended. In some cases, especially for synchronous short-running transactions, programmatic Data Objects may be used for performance improvement. However, the Data Objects better be derived or generated from the XML Schema, which would provide metadata enforcement (related utilities are available in Java and C# and are known as XML binding).
Define and lock down service interfaces and interfaces to the service resources such as persistence storage. Design the interface as business-oriented coarse-grained so your consumers can avoid frequent refactoring.
Best practices recommend avoiding RPC-like style in service interfaces. Web Services, RPC encoding, and SOAP-encoded arrays, in particular, seriously restrict service interoperability; consider a document/literal style instead. For non-XML cases, use a container style of interface to pass data structures and "exchanged" operations (aka commands in the Command Pattern).
It's important to design service interfaces with an eye to security, transaction, manageability, error handling, versioning, and interoperability from the beginning. Adding any of these things later is either extremely expensive (in effort, time, and money) or practically impossible.
For example, security is very important these days because of multiple technology threats and industrial/corporate security policies. Transaction support is dictated by its potential participation in service compositions. Manageability and error handling are core to any business service and process. Interoperability is one of the most critical factors in the SOA service lifecycle. That is, use of specific programming platform constructs can significantly reduce the utility of a service and lead to its refusal.
In spite of any ideas about keeping the service interface immutable, in real life variations in service invocation always happen over time. Anyway, a service interface isn't the only one to constitute a SOA service; there are also possibilities of SOA service behavior changes that affect service consumers. That is, the service has to be versioned. The versioning may be expressed via the interface and/or provided by a service registry. Only one condition has to be preserved in this regard - no service parts including the interface may be versioned separately from the service itself for service consumers.
A service access protocol may be considered a part of the service interface definition or a separate issue. The spectrum of communication protocols used in SOA is wide including TCP/IP, HTTP/HTTPS, SMTP, and IIOP.
The designer has to choose communication protocol(s) for every SOA service. When considering a protocol, one has to take into account: 1) invocation model - synchronous or asynchronous; 2) the amount of transferred data; 3) performance; 4) extendibility; 5) scalability; 6) reliability; 7) transaction; and 8) manageability.
Protocols are also important for communicating with service resources. It's better to use standardized protocols where possible. For example, current best practice refers to the Web Services Invocation Framework (WSIF) as a binding bridge between Web Service's WSDL and a particular service resource protocol while Java components rely on JBI.
The right protocol is a balanced choice. For example, HTTP/SOAP-based protocol is slower than IIOP but it's more extendible; TCP/IP is more reliable than UDP but it's slower. Don't forget that one service may have more than one communication channel. For example, the same session EJB can be reached through RMI-over-IIOP and SOAP-over-HTTP/S.
The pioneers of SOA implementations have recommend doing a "protocol performance POC" for particular kinds of tasks. That is, the earlier in the design process a performance test gives the results the better for the entire project because refactoring in this area means costly redesign from the scratch.
Define the SOA Service contract and Execution Context. One of the most important parts of the SOA service design is the definition of the SOA service contract template. While SOA RM simply says that "a contract...represents an agreement by two or more parties. Like policies, agreements are also about the conditions of using a service; they may constrain the expected real-world effects of using a service," the SOA service contract may be relatively complex. The contract, preferably written in XML, defines among other things:
- runtime policies/constraints
- runtime functional service parameters exposed to the service consumers; these parameters can be monitored and controlled
- non-functional service parameters like scheduled service availability
- business functional characteristics promised to service consumers like gathering audit information on service invocation events (the characteristics should be evidential and measurable)
- all service interfaces and communication protocols exposed to a particular consumer
- definitions or references to the Change Control procedures related to the service
- service version information
- references to the external (to your organization) SOA services invoked by the service. It's important for the business to know what other SOA services are involved because some political and business considerations may cause certain constraints on such involvement
- some custom or business domain specific parameters if needed
Runtime policies better be derived from the SOA service contract. The policies have to follow standards wherever possible. As SOA RM defines it, "A service contract is a measurable assertion that governs the requirements and expectations of two or more parties. Unlike policy enforcement, which is usually the responsibility of the policy owner, contract enforcement may involve resolving disputes between the parties to the contract. The resolution of such disputes may involve appeals to higher authorities."
The SOA service implementation process treats the service contract as an objective set of development tasks. Not all implementation tasks ought to be expressed in the service contract, but only those that get visibility to the service consumers. Other tasks are the subject of regular application requirements.
Another fundamental characteristic of a SOA service is the execution context. According to SOA RM, the "Execution context is central to many aspects of a service interaction." The context includes concepts such as the SOA service contract, and behavioural and information models. The same service can be viewed and act differently in different execution contexts. For example, a logging service may store data "in the clear" when serving load-balancing calculations but it has to encrypt data for financial risk calculations (i.e., financial regulations constitute the execution context).
Develop for security. SOA security isn't included in the SOA RM. Nonetheless it's obvious that the model's "real-world effects" can't survive without a business trust, i.e., without service security. We don't recommend developing a special implementation for each service. Instead use corporate security services at the level of service communication and service components, e.g., security policy syndication services.
Since a lot of security threats come from inside an organization, perimeter protection (external firewalls) isn't enough. We can outline the minimal SOA security requirements as 1) end-user bi-directional authentication with the service, and 2) inter-service bi-directional authentication. End-user and inter-service authorizations are much desired but may be optional in particular cases (e.g., a consumer may access everything after he's confirmed as a legitimate user). Other security measures are optional as well and depend on the nature of business data and processes.
Traditional security means like PKI, Kerberos, SSL, two-way SSL, user name and password tokens serve well in SOA. The service-specific means relate to the XML security standards. They include XML Digital Signature, XML standards for PKI (XKMS), SAML (for propagating the subject and its credentials), XACML (for transferring security policies and policy validation controls), and other encryption techniques applied to the exchange messages and communication channels. The choice of security methods to be applied depends on corporate and industry policies and regulations.
The takeaway here is that whatever security means ought to be implemented, they have to be considered in the design from the beginning. It's almost impossible or extremely expensive to add security in later phases of development.
Develop to support transactions. SOA RM recognizes the importance of service transaction as "higher-order attributes of services' process models" but doesn't detail the topic. In practice, SOA uses commonly accepted standards for transaction management on different platforms and adds cross-platform transaction management standards. Transactions can be used as in an aggregated service (choreographed aggregation) as in an orchestration of services reflecting fragments of Business Process Management (BPM). The orchestration is typically based on the BPEL standard that introduces short-running and long-running transactions.
A short-running transaction is usually a synchronous RPC-type invocation running according to the known rules of transaction isolation. Transaction standards for Web Services and messaging include WS-Transaction and WS Reliable Messaging (WS-RM). A short-running transaction is good for performance and commonly used for co-located services or in cases where failure is unlikely and error handling is simple. A long-running transaction is an asynchronous transaction where the transactional state may be persisted for long periods of time (days, months, years) and are restored after a certain event. This kind of transaction is quite useful in manual operations or human decision making in the automated business process. BPEL also introduces a transaction capable of "undoing" the mistaken results of the previous transaction by compensating them (like an overcharge refund).
Transactional behaviour has to be embedded in the SOA service from the design phase. This will help provide future service aggregation- and orchestration-based SOA solutions that might not be clear at the moment the service is initially designed.
Design for interoperability. As mentioned, interoperability is a crucial and critical characteristic of a SOA. The SOA RM says, "The value of SOA is that it provides a simple scalable paradigm for organizing large networks of systems that require interoperability to realize the value inherent in the individual components."
The recommendation here is to put a lot of attention on the data types and naming convention policies (besides the protocols) on the both sides of the interface - for the requester and the provider components. The policies must enforce compatibility and interoperability. One possible mechanism for data type and naming space interoperability control is XML Schema or any other schemas (metadata) accepted by the requester and the provider.
Another best practice recommendation is the creation of a proxy (isolation mapping layer) around the interfaces. That is, the data formats passed through the service interface shouldn't be driven by internal data formats (for either requester or provider). If the evolution of either side causes data format changes, it's better to create object mapping in the proxy than to try to modify the interface.
Use Rules for flexibility and easy-to-change implementation. Rule engines are just tools used in SOA infrastructure implementation and certainly not a part of the SOA RM. However, being combined with policies on one side and decision-making points on the other side, rule engines become quite powerful instruments for providing a high level of flexibility and quick adaptation to business changes. Leading SOA infrastructure/framework vendors include different rule engines into their business process design tools and Enterprise Service Bus solutions.
While rule engines are really convenient instruments, there's no compatibility between engines from different vendors; a rule engine locks you into a vendor. Anyway, not everything should be driven by rules - rule maintenance is expensive and requires special skills. That is, the designer and developer have to be reasonable in using rule engines.
For example, if the business logic introduces a point for choosing from a finite number of cases (while cases change seldom), using rules may be overkill. Another known risk in using rules is a potential rule contradiction. That is, a rule engine generally doesn't automatically validate if a new rule contradicts any existing rules and any permitted combinations of those rules. Without such validation, the new rule can bypass restrictive rules and expose prohibited information (e.g., a junior clerk gets access to a high-volume account). So when using a rule engine in SOA, proceed with caution.
Artificial Intelligence has the potential to massively disrupt IoT. In his session at 18th Cloud Expo, AJ Abdallat, CEO of Beyond AI, will discuss what the five main drivers are in Artificial Intelligence that could shape the future of the Internet of Things. AJ Abdallat is CEO of Beyond AI. He has over 20 years of management experience in the fields of artificial intelligence, sensors, instruments, devices and software for telecommunications, life sciences, environmental monitoring, process...
May. 2, 2016 05:15 PM EDT Reads: 958
Increasing IoT connectivity is forcing enterprises to find elegant solutions to organize and visualize all incoming data from these connected devices with re-configurable dashboard widgets to effectively allow rapid decision-making for everything from immediate actions in tactical situations to strategic analysis and reporting. In his session at 18th Cloud Expo, Shikhir Singh, Senior Developer Relations Manager at Sencha, will discuss how to create HTML5 dashboards that interact with IoT devic...
May. 2, 2016 05:00 PM EDT Reads: 1,069
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, will provide tips on how to be successful in large scale machine lear...
May. 2, 2016 03:45 PM EDT Reads: 1,325
The IoTs will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm and share the must-have mindsets for removing complexity from the development proc...
May. 2, 2016 03:00 PM EDT Reads: 287
SYS-CON Events announced today that Ericsson has been named “Gold Sponsor” of SYS-CON's @ThingsExpo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. Ericsson is a world leader in the rapidly changing environment of communications technology – providing equipment, software and services to enable transformation through mobility. Some 40 percent of global mobile traffic runs through networks we have supplied. More than 1 billion subscribers around the world re...
May. 2, 2016 02:45 PM EDT Reads: 1,058
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
May. 2, 2016 02:00 PM EDT Reads: 922
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus inter...
May. 2, 2016 12:30 PM EDT Reads: 1,267
You deployed your app with the Bluemix PaaS and it's gaining some serious traction, so it's time to make some tweaks. Did you design your application in a way that it can scale in the cloud? Were you even thinking about the cloud when you built the app? If not, chances are your app is going to break. Check out this webcast to learn various techniques for designing applications that will scale successfully in Bluemix, for the confidence you need to take your apps to the next level and beyond.
May. 2, 2016 12:00 PM EDT Reads: 1,541
There is an ever-growing explosion of new devices that are connected to the Internet using “cloud” solutions. This rapid growth is creating a massive new demand for efficient access to data. And it’s not just about connecting to that data anymore. This new demand is bringing new issues and challenges and it is important for companies to scale for the coming growth. And with that scaling comes the need for greater security, gathering and data analysis, storage, connectivity and, of course, the...
May. 2, 2016 11:45 AM EDT Reads: 934
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
May. 2, 2016 10:00 AM EDT Reads: 2,621
In his session at @ThingsExpo, Chris Klein, CEO and Co-founder of Rachio, will discuss next generation communities that are using IoT to create more sustainable, intelligent communities. One example is Sterling Ranch, a 10,000 home development that – with the help of Siemens – will integrate IoT technology into the community to provide residents with energy and water savings as well as intelligent security. Everything from stop lights to sprinkler systems to building infrastructures will run ef...
May. 2, 2016 10:00 AM EDT Reads: 1,015
Digital payments using wearable devices such as smart watches, fitness trackers, and payment wristbands are an increasing area of focus for industry participants, and consumer acceptance from early trials and deployments has encouraged some of the biggest names in technology and banking to continue their push to drive growth in this nascent market. Wearable payment systems may utilize near field communication (NFC), radio frequency identification (RFID), or quick response (QR) codes and barcodes...
May. 2, 2016 09:15 AM EDT Reads: 857
The increasing popularity of the Internet of Things necessitates that our physical and cognitive relationship with wearable technology will change rapidly in the near future. This advent means logging has become a thing of the past. Before, it was on us to track our own data, but now that data is automatically available. What does this mean for mHealth and the "connected" body? In her session at @ThingsExpo, Lisa Calkins, CEO and co-founder of Amadeus Consulting, will discuss the impact of wea...
May. 2, 2016 08:45 AM EDT Reads: 783
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
May. 2, 2016 06:30 AM EDT Reads: 2,532
The IoT has the potential to create a renaissance of manufacturing in the US and elsewhere. In his session at 18th Cloud Expo, Florent Solt, CTO and chief architect of Netvibes, will discuss how the expected exponential increase in the amount of data that will be processed, transported, stored, and accessed means there will be a huge demand for smart technologies to deliver it. Florent Solt is the CTO and chief architect of Netvibes. Prior to joining Netvibes in 2007, he co-founded Rift Technol...
May. 1, 2016 11:00 PM EDT Reads: 1,612
We’ve worked with dozens of early adopters across numerous industries and will debunk common misperceptions, which starts with understanding that many of the connected products we’ll use over the next 5 years are already products, they’re just not yet connected. With an IoT product, time-in-market provides much more essential feedback than ever before. Innovation comes from what you do with the data that the connected product provides in order to enhance the customer experience and optimize busi...
May. 1, 2016 05:00 PM EDT Reads: 1,066
SYS-CON Events announced today that Stratoscale, the software company developing the next generation data center operating system, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Stratoscale is revolutionizing the data center with a zero-to-cloud-in-minutes solution. With Stratoscale’s hardware-agnostic, Software Defined Data Center (SDDC) solution to store everything, run anything and scale everywhere...
May. 1, 2016 01:30 PM EDT Reads: 1,601
Angular 2 is a complete re-write of the popular framework AngularJS. Programming in Angular 2 is greatly simplified – now it's a component-based well-performing framework. This immersive one-day workshop at 18th Cloud Expo, led by Yakov Fain, a Java Champion and a co-founder of the IT consultancy Farata Systems and the product company SuranceBay, will provide you with everything you wanted to know about Angular 2.
May. 1, 2016 01:00 PM EDT Reads: 1,773
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
May. 1, 2016 12:15 PM EDT Reads: 2,354
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
May. 1, 2016 11:00 AM EDT Reads: 1,118