Click here to close now.

Welcome!

JAVA IoT Authors: Sematext Blog, Roger Strukhoff, Elizabeth White, Carmen Gonzalez, XebiaLabs Blog

Related Topics: Microservices Expo, JAVA IoT, Industrial IoT

Microservices Expo: Article

SQL Transparent Hierarchical Processing of Relational, XML and IMS Data

Every ANSI SQL processor contains a powerful hierarchical processor

Current SQL support of relational, XML and hierarchical legacy data such as IMS is driven by flattening the hierarchical data in order to integrate it naturally with relational (flat) data so that it can be processed relationally. Unfortunately, this strips out the natural semantics in hierarchical data which has the capability to dynamically increase the value of the data being processed and to perform powerful hierarchical operations.

The SQL-92 standard introduced the LEFT Outer Join which offers a powerful alternative to standard relational processing that can be used to perform full hierarchical processing naturally and inherently. This enables SQL to seamlessly and transparently integrate relational data at a full hierarchical structured data processing level with XML, IMS and other forms of legacy hierarchical data. This means there is no data loss for the integration of hierarchical and relational data while increasing the level of powerful automatic hierarchical operations.

It is important to note that current SQL access to XML via the SQL/XML standard is through XML semistructured data processing. This is a very relaxed hierarchical principled fuzzy process for processing markup and requires user specified data structure navigation. The natural navigationless hierarchical processing performed by SQL shown in this article always following precise hierarchical principles and restrictions. This keeps the hierarchical structures unambiguous for automatic processing. This will all  be covered in this article.

Also significant to SQL full hierarchical processing is its automatic support of multipath processing. This was a powerful hierarchical processing used for querying IMS databases four decades ago before the advent of relational processing.This utilizes the naturally occurring semantics in multipath queries to transparently perform powerful nonlinear hierarchical processing. This enables non technical users to easily specify more powerful internally complex dynamic queries than are available today. This article will explain and demonstrate this advanced SQL hierarchical multipath processing capability and its advantages that break away from the current relational flat mindset and its limitations. The SQL hierarchical processing examples described in this article have been tested on a hierarchical processing middleware prototype which uses a standard ANSI SQL processor as its hierarchical processing engine.

Basic Hierarchical Data Modeling
The following SQL view definition in Figure 1 uses a sequence of standard SQL LEFT Outer Joins to model the shown hierarchical structure. This hierarchical view is modeled at the node level to define basic physical structures like IMS or XML. It also defines logical hierarchical structures using flat data like relational. The LEFT Outer Join hierarchically preserves the left data argument over the right argument. This allows the left data argument to exist when there is no matching right data argument. These LEFT Outer Joins can be strung together along with the ON clauses to define and build full hierarchical multiple path structures.

The ON clause was also introduced in the ANSI SQL-92 Outer Join to offer more local control over the WHERE clause for specifying join criteria. Notice in Figure 1 that the ON clause is specified at each specific join point to control its specific join operation. All of these capabilities allow the LEFT Outer Join syntax to exactly model hierarchical structures unambiguously. This was not previously possible using a single WHERE clause for join control. The WHERE clause is still used to specify global data filtering described later.

    Basic Hierarchical Data Modeling

    CREATE VIEW RDB AS
    SELECT * FROM R
    LEFT JOIN D ON R.r=D.d /* Extend path from node R to node D */
    LEFT JOIN B ON R.r=B.b /* Start new Path from node R to node B */

    Basic Structure
         R
        /  \
      D   B

   Figure 1: SQL Hierarchical View

The corresponding hierarchical semantics to the LEFT Outer Join data modeling syntax shown in Figure 1 defines exactly how this data structure definition operates hierarchically. It performs hierarchical data preservation when processed directly by the ANSI SQL processor. As shown in Figure 1, this hierarchical data definition can be specified in an SQL view. The R, D, and B items in the hierarchical view structure in Figure 1 are relational table names which also represent nodes in the hierarchical structure.

The hierarchical structure defined in Figure 1 above follows basic hierarchical principles supplied by the SQL Outer Join semantics. Nodes D and B can not exist without the higher level R node. But node R can exist without nodes D and B. Node D can exist without node B and visa versa because hierarchical pathways are independent of each other.

Static Hierarchical Structure Combining
Basic structure views like the one in Figure 1 above can be joined together hierarchically to create larger more powerful views that combine hierarchical structures. This is also performed by using the LEFT Outer Join and ON clause as shown in Figure 2 below. The only difference is that the view structures are being joined instead of isolated nodes. In this example, two additional basic views, IMS and XYZ, are used. They represent a physical IMS and XML data source modeled in the same way as the logical RDB view was defined in Figure 1 by using the LEFT Outer Join. In the IMS view, Segments I, M, and S represent nodes. In the XML view, Elements X, Y, and Z represent nodes. These common nodes allow seamless heterogeneous hierarchical data structure integration. Not being relational data, the non relational views contain additional specific physical meta information needed for their specific physical processing. They will also require an external process for retrieving their non relational data into rowsets when needed. These processes will be covered further later.

      Logical Data Structure

      CREATE VIEW TestView AS
      SELECT * FROM RDB
      LEFT JOIN IMS ON B.b=I.
      LEFT JOIN XYZ ON B.b=X.x

     Full Heterogeneous Structure

           R
          /   \
        D    B
             /   \
           I      X
          / \     / \
        M  Y Z

  Figure 2: Logical Data View

The combined view named TestView in Figure 2 above consists of views RDB, IMS and XYZ. This makes TestView a heterogeneous view consisting of multiple data structure types defined seamlessly to ANSI SQL by using the LEFT Outer Join data modeling operation described above. When joining physical view structures together, the result is a logical structure. Logical structures can be joined with other logical or physical structures. Logical structures can also be directly processed by the ANSI SQL processor. This is because the logical view defined by TestView in Figure 2 with its corresponding relational rowset hierarchically maps to the combined and expanded LEFT Outer Join views RDB, IMS and XYZ. This supplies the natural hierarchical data modeling syntax and semantics needed for SQL to automatically perform hierarchically on the now homogeneous flattened rowset.

While the hierarchical data has been flattened in the rowset, it still automatically represents hierarchically preserved multiple variable length pathways using null filled data. The null data is used to fill out the variable length paths keeping the data aligned properly in the rowset. This allows hierarchical structures to be still operated on hierarchically by SQL. These views are temporarily materialized when processed, so they do not take up space when not used and avoid replicated data problems.

Global Hierarchical Data Filtering
As already mentioned, the newer ON clause is used instead of the WHERE clause for the purpose of specifying separate and specific uses of the join criteria. The WHERE clause operation is still responsible for data filtering. The difference between the ON clause and the WHERE clause is that the ON clause operation is local to its position in the structure affecting only pathways linked below it, while the WHERE clause filtering operation is global, operating on the entire structure. This means the entire hierarchical structure is processed as shown by the flow direction arrows in Figure 3 below. In Figure 3, node B is filtered directly by the WHERE clause. This can affect the data occurrences above and below node B removing data occurrences of nodes R and X. Notice also that being a hierarchical structure, node D a cousin relationship of node B, is also affected because if an R node data occurrence is filtered out, so is its descendant node D data occurrences. This hierarchical operation can cascade which can affect the entire global structure.

 

    Hierarchical Data Filtering Direction

    R        SELECT R.r, D.d, B.b, X.x
   ↓        FROM TestView
   D B ßWHERE B.b=5
       ↓
       X

  Figure 3: Global Hierarchical Data Filtering

The global hierarchical filtering operation in Figure 3 above is taken care of automatically in SQL because entire rows in rowsets are filtered out by the WHERE clause. When performing the more flexible and data preserving Outer Join ON clauses, only portions of rows are affected. These are the pathways linked below the ON clause being processed as mentioned above. The SQL ON clause operates while the hierarchical data structures are being built, while the WHERE clause operates  after the data structure is complete. In this way, the entire structure is available to operate on. Optimization does allow concurrent WHERE clause processing as long as the result is the same.

This data filtering example is a very simple case. When multiple data items are used in data filtering, the processing is still automatic and correct. It becomes more internally complex and more complicated to explain. For example, this can be seen in the filtering criteria of: WHERE B.b=5 OR D.d=4 which is different than WHERE B.b=5 AND D.d=4. The OR condition combines the results of each data filter. The AND condition combines the results of each filter when both sides find a match otherwise no data is selected. This operation occurs for each data occurrence. With this complex WHERE clause processing, it may be easier to grasp this operation as data qualification. Data filtering and data qualification are the same.

Hierarchical Data Processing Optimization
By supporting only hierarchical structured data which is necessary for standard SQL processing, a powerful generic semantic hierarchical optimization can be applied by middleware before the standard relational optimization is performed. This hierarchical optimization is useful because it will not be fully recognized by the standard relational optimization. This hierarchical optimization will dynamically remove all unnecessary pathways from a queried SQL hierarchical view or virtual heterogeneous view. Because of hierarchical data preservation principles, only referenced nodes including the root node and their intervening nodes need to be accessed. Since ON clause data references are part of the structure definition process, they are not considered query references for this optimization and do not influence optimization.

This hierarchical optimization is demonstrated in the query example in Figure 4 below where nodes D, X, Y, and Z are not specified on the SELECT list and are not accessed without changing the hierarchical semantics of the query. This hierarchical optimization also has a positive effect on the following relational optimization by enabling it to operate on a smaller structure making its operation easier to recognize optimizations.

 

       SELECT R.r, I.i, M.m 
       FROM TestView 
       WHERE S.s=5

   TestView   Optimized 
   Structure   Structure

      R               R
      / \                |
    D B              B
       /  \     è | 
     I     X           I 
    / \    / \          \
  M S  Y Z     M   S

  Figure 4: Hierarchical Optimization

Notice in Figure 4 above that node B was not referenced in the query, but required access because it is on the path to a referenced node I keeping the semantics of the structure accurate. Hierarchical optimization greatly reduces data access and relational data explosions without changing the hierarchical structure semantics and output. Standard Inner Joins can not use this optimization because they can not define hierarchical structures with their lack of data preservation. In addition, Inner Join use will destroy existing hierarchical structures. Most queries are still using the older Inner Join operation as their primary join operation. This means the Inner Join operation should not be used with hierarchical structures because it is missing the needed hierarchical processing principles.

Global Views and Their Usage
The additionally supplied hierarchical optimization described in Figure 4 above allows all views to be global views because there is no overhead for unused nodes in a dynamic view execution. This allows the use of larger views making much fewer views possible. This makes queries easier for the user to specify. This is because the specific view knowledge is not necessary for the user to specify efficient queries. This also reduces or eliminates join operations by the user because they are abstracted (hidden) in the global view.

Variable Processing and Output Data Structure Aggregation
SQL's powerful variable SELECT list operation allows its hierarchical optimization to be dynamically and automatically controlled from query to query by its flexible SELECT list. This also enables automatic dynamic tailoring of the necessary processing from query to query. A view invoked with a variable SELECT list can specify entirely different queries specifically tailored to the desired output as shown in Figure 5 below. This example is actually a continuation of the query shown in Figure 4 above to show its third and final state, this is its result structure. This automatic variable processing and dynamic output structure flexibility offers advanced new capabilities and uses such as dynamic publication support.

       SELECT R.r, I.i, M.m
       FROM TestView
       WHERE S.s=5

  Optimized    Result
   Structure   Structure

      R              R
       |                |
      B      è I
       |                | 
      I               M
     / \
   M S

   Figure 5: Variable Processing with Output Aggregation

In Figure 5 above, the hierarchical result structure is shown which contains only the dynamic SELECT list's specified nodes. This has the effect of dynamically controlling the desired output structure. Notice that the S node used in the WHERE clause was not output because it was not selected for output which is valid. Also note that the B node that was not referenced, but was necessary for internal database navigation, was not output as expected. This had the output result of closing the gap between nodes R and I. This is normal in hierarchical processing and is called node promotion which aggregates (condenses) the result nicely around the desired output data. This automatic reconnecting of pathways correctly preserves the remaining hierarchical semantics of the structure. Interestingly this is also standard in relational processing where it is known as projection produced naturally by the relational SELECT operation. So this mapping is both relationally and hierarchically accurate.

Multipath Query Data Qualification
Specifying multiple paths on a data qualification query requires nonlinear processing logic. These multipath queries using the SQL WHERE operation are incredibly powerful and extremely complex to process. They correlate references across pathways that interact with each other. This processing is hidden by SQL's automatic and inherent hierarchical structured data processing capability.

There are two different usages of a SQL multipath query data qualification. These are: 1) Selecting data from one pathway of the hierarchical data structure based on data from another pathway or: 2) WHERE clause searches comparing data on or across multiple pathways. These multipath queries use the naturally occurring structure semantics that exists between the concurrently processed pathways to solve this query. This significantly increases the number of different queries possible. This enables a new level of powerful hierarchical processing available to uncover new more deeper meaning in the data without requiring additional query syntax or knowledge of the structure for the user. The previously mentioned global view capability magnifies the usefulness of these multipath processing capabilities further by making unlimited combinations of pathways more available and easier to specify.

Dynamic Hierarchical Structure Combining
It is important to point out that SQL processing can perform all of its hierarchical processing capabilities dynamically. This means that SQL can dynamically and hierarchically join hierarchical structures in an ad hoc or interactive way (instead of limited to a static view). This is shown below in Figure 6 below where views RDB and IMS are dynamically joined at runtime to combine structures.

 

    SELECT B.b, M.m, S.s 
    FROM RDB
    LEFT JOIN IMS ON B.b=I.i

    Result Structure

       B
      /  \
    M   S

    Figure 6: Dynamic Structure Joining

The example of node promotion in Figure 6 above also demonstrates node collection occurring. This can be seen when nodes M and S from separate pathways collect under the common B node. Notice that node promotion can occur across the entire virtual structure. In this case, node promotion occurs across a relational and IMS structure.

Extending SQL's Inherent Hierarchical Processing
To fully utilize SQL's inherent hierarchical processing correctly, the SQL must be specified hierarchically as described in this article. For SQL to control automatic I/O capabilities that use non relational access, SQL must be extended. This extension automatically uses the structure information supplied from the Left Outer Join hierarchical structure modeling and the access meta information supplied in new non relational views describe previously. For testing and proof of concept, this extended operation has been isolated and provided in a hierarchical processing enabler middleware prototype. It operates on top of ANSI SQL processors to support hierarchical data types such as IMS and XML to automatically and seamlessly enable this inherent hierarchical processing capability in the user's SQL processor. This extended capability supplied by the hierarchical processing enabler middleware is described in the next section.

The ANSI SQL Hierarchical Middleware
The hierarchical processing middleware prototype uses a standard ANSI SQL processor as the full multipath hierarchical processing engine. It is required to seamlessly power the hierarchical middleware prototype. The middleware consists of a Preprocessor, Post Processor, and Non Relational Data Access Backend components shown below in Figure 7 below. This middleware supports the new capabilities needed to extend SQL's natural hierarchical processing to non relational data such as XML and IMS including other legacy data such as VSAM and IDMS.

The Preprocessor automatically analyzes the LEFT Outer Join SQL source input to dynamically determine the virtual hierarchical structure to be processed. This also makes hierarchical structure-aware processing available to the other following phases of the middleware. The SQL input describing the data structure is hierarchically optimized to remove unneeded node pathways for the active query by analyzing the hierarchical structure and SELECT list as previously described. The modified optimized SQL is then submitted to the standard ANSI SQL Processor for full multipath level hierarchical processing.

The Real-time Non Relational Data Access Backend is used to retrieve non relational data and convert it into a hierarchically preserved relational rowset for the ANSI SQL Processor. It is invoked when a needed non relational data type is requested from the ANSI SQL Processor using an external table or UDF call. This external table or UDF mechanism will include pointers to the non relational processing information specified from the non relational hierarchical view and runtime optimization information. This enables a hierarchically preserved relational rowset to be returned efficiently by the backend which is hierarchically mapped into the LEFT Outer Join defined for this non relational view. This makes its SQL hierarchical processing seamless.

The Post Processor takes the completed relational result set from the ANSI SQL Processor removing the relationally introduced replicated data from the result and converts the result to the desired hierarchical structured output format. Duplicate and replicated data are treated differently. Duplicate data is meaningful and is preserved by uniquely tagging it on input; this also preserves the data order if necessary, such as for XML. The output structure format was determined from the structure information extracted in the Preprocessor. These three additional sections added for hierarchical support that make up the hierarchical middleware prototype are shown mapped around the ANSI SQL processor as shown Figure 7 below.

                                 Hierarchical Processing Middleware Design

                                             SQL Source
                                                    ↓
                                           Preprocessor
   Real-time Non                        
   Relational Data çè ANSI SQL Processor Performing Hierarchically
   Access Backend                     
                                          Post Processor
                                                    ↓
                                   XML Hierarchical Output

           Figure 7: Hierarchical Processing Middleware

The hierarchical processing middleware prototype laid out in Figure 7 produces hierarchically formatted XML output automatically which can be easily changed to produce any type of hierarchical data structure output. The hierarchical processing middleware has proven the powerful ability of operating directly on top of in place ANSI SQL processors at the customer's site to automatically support transparent hierarchical processing. This description of the hierarchical processing middleware should indicate how the Preprocessor, Real-time Non Relational Data Access Backend, and Post Processor functions can be moved into the ANSI SQL processor as an alternative implementation.

Benefits of Full  Multipath Hierarchical Processing
SQL's ability to naturally perform full multipath hierarchical processing has many new benefits listed below. This inherent automatic hierarchical processing allows the dynamic access of legacy data at a hierarchical level. With this capability and the beforehand knowledge of the entire query processing necessary derived from the preprocessor step described previously, extremely optimized dynamic data access is possible. This can be optimally applied to IMS data access by utilizing its database multiple positioning and dynamic SSA (Segment Search Argument) construction used for database navigation. This advanced hierarchical processing of IMS can be integrated with relational and tabular data as well as other hierarchical databases and data such as XML at a seamless hierarchical processing level. This can be performed on mainframes to supplement IMS data or it can be used to supplement Internet XML data with seamless and transparent access of IMS data. New features and capability include:

  • Improved SQL performance when processing IMS by using hierarchical optimization
  • Increases value of the data by utilizing the natural existing hierarchical structure semantics
  • Improves processing correctness and accuracy using principled hierarchical processing
  • Increases processing capabilities to include advanced hierarchical processing capabilities
  • Improved ease of use for hierarchical processing requiring no technical knowledge to use
  • Extends SQL navigationless structured data support to XML processing missing today

Conclusion
This article explained how SQL can now perform full hierarchical processing and what the advantages and new capabilities are and how they operate. The automatic utilizing of the natural hierarchical structure semantics can perform many new capabilities as described in this article. SQL hierarchical queries can now specify more powerful data value increasing queries that freely specify multiple pathways allowing more internally complex processing to be performed inherently. These multipath queries require special processing known as LCA logic which is performed automatically by SQL to control the range of hierarchical processing across pathways to keep the result meaningful and correct. See the Multipath LCA Logic sidebar for more information on this. The SQL hierarchical processing shown in this article can be tested using the hierarchical processing prototype available for online interactive testing at: www.adatinc.com/demo.html. The previous article published under this topic also included a set of powerful automatic hierarchical data value increasing capabilities.

 

Multipath Lowest Common Ancestor (LCA) Logic [Sidebar]

Multiple path processing requires logic involving the hierarchical distance between the referenced pathways data to determine the range of processing in order to keep the result meaningful. This range of processing is controlled by Lowest Common Ancestor (LCA) logic. It uses the LCA node between referenced pathways shown in Figure 6 to define the range of control to keep the result meaningful. Too high a level introduces meaningless data into the result, too low a level will miss meaningful data. This LCA processing logic is very complex and is not practical to be performed manually and procedurally. Fortunately, we found SQL's inherent hierarchical processing performs this LCA processing indicated below in  Figure Sidebar1 automatically as part of its relational Cartesian product processing when performing hierarchical data processing.

     SELECT R.r, M.m, D.d
     FROM TestView
     WHERE Y.y=1 AND Z.z=2

       Processed Structure

      R ß SELECT D.d WHERE Z.z=2 --- LCA Node =R 
     /  \
   D   B ß SELECT M.m WHERE Z.z=2 --- LCA Node = B
        / \
      I    X ß WHERE Y.y=2 and Z.z=1 --- LCA Node = X 
     / \   / \
   M S Y Z

   Figure Sidebar1: Lowest Common Ancestor (LCA) Processing

Figure Sidebar1 above shows SQL's different LCA uses. The WHERE clause in this example has a compound data filtering condition that spans two pathways on nodes Y and Z. This means that all combinations of the two conditions across the different paths need to be tested under the current LCA X node data occurrence until a match is found. The SELECT LCA operation involves the use of the SELECT operation with WHERE filtering as in SELECT D.d WHERE Z.z=2 taken from the example in Figure 6. The D and Z node references produce the LCA node of R. This operation will select all D.d data occurrences under the current LCA R node data occurrence when Z.z=2 is true. The SELECT operation can use a different LCA for each of its data items as in SELECT M.m, D.d WHERE Z.z=2. It produces LCA node B for SELECT data item M.m, and produces LCA node R for SELECT data item D.d. Current hierarchical querying of hierarchical data outside of SQL does not automatically support multipath LCA processing and it is too complicated to specify manually.

More Stories By Michael M David

Michael M. David is founder and CTO of Advanced Data Access Technologies, Inc. He has been a staff scientist and lead XML architect for NCR/Teradata and their representative to the SQLX Group. He has researched, designed and developed commercial query languages for heterogeneous hierarchical and relational databases for over twenty years. He has authored the book "Advanced ANSI SQL Data Modeling and Structure Processing" Published by Artech House Publishers and many papers and articles on database topics. His research and findings have shown that Hierarchical Data Processing is a subset of Relational Processing and how to utilize this advanced inherent capability in ANSI SQL. Additionally, his research has shown that advanced multipath (LCA) processing is also naturally supported and performed automatically in ANSI SQL, and advanced hierarchical processing operations are also possible. These advanced capabilities can be performed and explained in the ANSI SQL Transparent XML Hierarchical Processor at his site at: www.adatinc.com/demo.html.

@ThingsExpo Stories
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Architect for the Internet of Things and Intelligent Systems at Red Hat, described how to revolutioniz...
The world is at a tipping point where the technology, the device and global adoption are converging to such a point that we will see an explosion of a world where smartphone devices not only allow us to talk to each other, but allow for communication between everything – serving as a central hub from which we control our world – MediaTek is at the heart of both driving this and allowing the markets to drive this reality forward themselves. The next wave of consumer gadgets is here – smart, connected, and small. If your ambitions are big, so are ours. In his session at @ThingsExpo, Jack Hu, D...
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, and physical persons. In the IoT vision, every new "thing" - sensor, actuator, data source, data con...
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York City, NY. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption by amplifying "faint signals" from the alpha geeks who are creating the future. An active participa...
We’re entering a new era of computing technology that many are calling the Internet of Things (IoT). Machine to machine, machine to infrastructure, machine to environment, the Internet of Everything, the Internet of Intelligent Things, intelligent systems – call it what you want, but it’s happening, and its potential is huge. IoT is comprised of smart machines interacting and communicating with other machines, objects, environments and infrastructures. As a result, huge volumes of data are being generated, and that data is being processed into useful actions that can “command and control” thi...
There will be 150 billion connected devices by 2020. New digital businesses have already disrupted value chains across every industry. APIs are at the center of the digital business. You need to understand what assets you have that can be exposed digitally, what their digital value chain is, and how to create an effective business model around that value chain to compete in this economy. No enterprise can be complacent and not engage in the digital economy. Learn how to be the disruptor and not the disruptee.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you'll have no problem fil...
There's Big Data, then there's really Big Data from the Internet of Things. IoT is evolving to include many data possibilities like new types of event, log and network data. The volumes are enormous, generating tens of billions of logs per day, which raise data challenges. Early IoT deployments are relying heavily on both the cloud and managed service providers to navigate these challenges. In her session at Big Data Expo®, Hannah Smalltree, Director at Treasure Data, discussed how IoT, Big Data and deployments are processing massive data volumes from wearables, utilities and other machines...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
The worldwide cellular network will be the backbone of the future IoT, and the telecom industry is clamoring to get on board as more than just a data pipe. In his session at @ThingsExpo, Evan McGee, CTO of Ring Plus, Inc., discussed what service operators can offer that would benefit IoT entrepreneurs, inventors, and consumers. Evan McGee is the CTO of RingPlus, a leading innovative U.S. MVNO and wireless enabler. His focus is on combining web technologies with traditional telecom to create a new breed of unified communication that is easily accessible to the general consumer. With over a de...
Disruptive macro trends in technology are impacting and dramatically changing the "art of the possible" relative to supply chain management practices through the innovative use of IoT, cloud, machine learning and Big Data to enable connected ecosystems of engagement. Enterprise informatics can now move beyond point solutions that merely monitor the past and implement integrated enterprise fabrics that enable end-to-end supply chain visibility to improve customer service delivery and optimize supplier management. Learn about enterprise architecture strategies for designing connected systems tha...
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, deploy, and manage applications integrating voice, video and data. He is the co-founder of TeleStax, a...
The Internet of Things (IoT) promises to evolve the way the world does business; however, understanding how to apply it to your company can be a mystery. Most people struggle with understanding the potential business uses or tend to get caught up in the technology, resulting in solutions that fail to meet even minimum business goals. In his session at @ThingsExpo, Jesse Shiah, CEO / President / Co-Founder of AgilePoint Inc., showed what is needed to leverage the IoT to transform your business. He discussed opportunities and challenges ahead for the IoT from a market and technical point of vie...
Grow your business with enterprise wearable apps using SAP Platforms and Google Glass. SAP and Google just launched the SAP and Google Glass Challenge, an opportunity for you to innovate and develop the best Enterprise Wearable App using SAP Platforms and Google Glass and gain valuable market exposure. In his session at @ThingsExpo, Brian McPhail, Senior Director of Business Development, ISVs & Digital Commerce at SAP, outlined the timeline of the SAP Google Glass Challenge and the opportunity for developers, start-ups, and companies of all sizes to engage with SAP today.
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges. In his session at @ThingsExpo, Jeff Kaplan, Managing Director of THINKstrategies, will examine why IT must finally fulfill its role in support of its SBUs or face a new round of...
One of the biggest challenges when developing connected devices is identifying user value and delivering it through successful user experiences. In his session at Internet of @ThingsExpo, Mike Kuniavsky, Principal Scientist, Innovation Services at PARC, described an IoT-specific approach to user experience design that combines approaches from interaction design, industrial design and service design to create experiences that go beyond simple connected gadgets to create lasting, multi-device experiences grounded in people's real needs and desires.