Welcome!

Java IoT Authors: Carmen Gonzalez, Liz McMillan, Pat Romanski, Elizabeth White, Mano Marks

Related Topics: Java IoT, Industrial IoT, Weblogic, Eclipse, Machine Learning , Recurring Revenue

Java IoT: Article

Column Store, In-Memory, MPP Databases and Oracle

How Oracle implements latest developments in database world

( For latest information on Oracle 12c database update please refer to the following article: Oracle 12c Database and How It Relates to SAP Hana )

RDBMSs are stable and mature products. While there is nothing radically new on horizon that would challenge Codd's relational theory and related advances in data processing there are some developments that force established vendors like Oracle to come up with new features and products.

Column Stores and Oracle
Column store concept has been around for quite a while. Vendors like HP Vertica grabbed some market share in data warehousing segment with their column store,  MPP databases. Oracle Exadata is offering Hybrid Columnar Compression -  solution that modifies Oracle standard row based storage (NSM, or row-major) into proprietary format that is probably closer to PAX (as opposed to DSM) classification. Rows of data are reorganized, broken down into columns, compressed and stored in Compression Units which consist of multiple Oracle data blocks. CU is physically implemented as a standard Oracle single column chained row. (Description of CU layout is based on marvelous article written by Oracle expert  Jonathan Lewis).

This is truly hybrid design, i.e., column store is implemented on top of a standard row store.

It is not in the scope of this article to discuss pros and cons of this implementation from performance, locking, compression and other points of view. I will just mention that HCC requires change in standard operating procedures and methods.

MPP (shared nothing) and Oracle
Column-based stores like HP Vertica use Multiple Parallel Processing, shared nothing design to enhance performance by bringing processing closer to data, i.e., data is processed in parallel on the node where it resides. Volume of data that is moved around is reduced with the additional benefit of CPU and data proximity.

Oracle's implementation of this idea could be classified as asymmetric MPP. Oracle Exadata uses offloading to storage layer, Smart Scan, Storage Indexes and other techniques to improve performance.

Storage layer (Exadata cells) are tasked with as much work as possible to reduce load on database server and network. Each Exadata storage cell has the ability to perform some parts of data processing operations as well as decompression.

IMDB and Oracle
SAP relatively recently released Hana - a fully functional in-memory RDBMS, targeted for both OLTP and OLAP applications. Hana operates on the premise that whole database is in memory and not on disk. Majority of data processing is now pointer based arithmetic, so whole sections of RDBMS code related to moving data back and forth between disk, RAM and CPU are not needed any more. This is all possible because memory is more affordable and abundant, so much so that most of modern OLTP databases can completely fit within modern server's RAM.

Oracle puchased Times Ten in-memory database, but marketed it mostly as caching layer to standard Oracle database. Times Ten is not marketed as stand alone IMDB the way SAP Hana is.

Oracle database can have Flash Cache devices configured as an extension of SGA for better performance (via database parameter), or for database logging purposes.

Exadata can be configured with terabytes of Flash Cache memory for database caching and to serve as solid state disk. This is not memory directly accessible by CPU though ( DRAM ), i.e., Oracle database accesses Flash Cache via PCI interface and IO operating system calls. In other words, Flash is treated the same as disk, with all negative consequences of such approach regarding code complexity and performance. The latest release of Exadata performs writes directly to flash cache first to improve performance. We should expect more optimizations that will try to better utilize abundance of various types of memory. Expected scenario could be similar to Microsoft Hekaton project is also about adding IMDB features to SQL Server ( tables can be loaded in memory and processed in IMDB fashion, with reduced latching and locking; perhaps choice between different storage engines will be possible).

Conclusion
Oracle will probably continue to execute on strategy that worked well in the past - gradual inclusion of new technologies into its core RDBMS product (like it did with programmable server, OODBMS, Internet database, XML, partitioning, etc.). None of the latest developments in server technologies and database world is as seismic as introduction of RDBMS, client-server and Internet computing was. Oracle was so far successful  in modifying its database engine to adjust to the changes in hardware and data processing methods. We expect this trend to continue, as pace of technological innovation is somewhat slowing down and no truly disruptive changes are on horizon. Oracle solutions are designed to introduce and take advantage of these new (old) technologies and avoid cannibalizing existing profits. Intent is to maximally protect existing legacy RDBMS software revenues and integrate and sell products that came with new hardware and software acquisitions. Oracle Exadata, for example, could be viewed just as intelligent, Oracle database aware and Oracle produced SAN, bundled with database server. It is perhaps safe strategy in an environment where even mediocre and lackluster repackaging, modifying and integrating acquisitions is not seriously challenged by radical new ideas or strong, competitive implementations of existing concepts and technologies. SAP Hana, for example, is also unification layer built on top of in-house built or acquired products.

More Stories By Ranko Mosic

Ranko Mosic, BScEng, is specializing in Big Data/Data Architecture consulting services ( database/data architecture, machine learning ). His clients are in finance, retail, telecommunications industries. Ranko is welcoming inquiries about his availability for consulting engagements and can be reached at 408-757-0053 or [email protected]

@ThingsExpo Stories
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
Discover top technologies and tools all under one roof at April 24–28, 2017, at the Westin San Diego in San Diego, CA. Explore the Mobile Dev + Test and IoT Dev + Test Expo and enjoy all of these unique opportunities: The latest solutions, technologies, and tools in mobile or IoT software development and testing. Meet one-on-one with representatives from some of today's most innovative organizations
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and E...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
The WebRTC Summit New York, to be held June 6-8, 2017, at the Javits Center in New York City, NY, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 20th International Cloud Expo and @ThingsExpo. WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web co...
"A lot of times people will come to us and have a very diverse set of requirements or very customized need and we'll help them to implement it in a fashion that you can't just buy off of the shelf," explained Nick Rose, CTO of Enzu, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex softw...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
The security needs of IoT environments require a strong, proven approach to maintain security, trust and privacy in their ecosystem. Assurance and protection of device identity, secure data encryption and authentication are the key security challenges organizations are trying to address when integrating IoT devices. This holds true for IoT applications in a wide range of industries, for example, healthcare, consumer devices, and manufacturing. In his session at @ThingsExpo, Lancen LaChance, vic...
Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, discussed the impact of technology on identity. Sho...
Manufacturers are embracing the Industrial Internet the same way consumers are leveraging Fitbits – to improve overall health and wellness. Both can provide consistent measurement, visibility, and suggest performance improvements customized to help reach goals. Fitbit users can view real-time data and make adjustments to increase their activity. In his session at @ThingsExpo, Mark Bernardo Professional Services Leader, Americas, at GE Digital, discussed how leveraging the Industrial Internet and...
What are the new priorities for the connected business? First: businesses need to think differently about the types of connections they will need to make – these span well beyond the traditional app to app into more modern forms of integration including SaaS integrations, mobile integrations, APIs, device integration and Big Data integration. It’s important these are unified together vs. doing them all piecemeal. Second, these types of connections need to be simple to design, adapt and configure...
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
WebRTC is about the data channel as much as about video and audio conferencing. However, basically all commercial WebRTC applications have been built with a focus on audio and video. The handling of “data” has been limited to text chat and file download – all other data sharing seems to end with screensharing. What is holding back a more intensive use of peer-to-peer data? In her session at @ThingsExpo, Dr Silvia Pfeiffer, WebRTC Applications Team Lead at National ICT Australia, looked at differ...
"ReadyTalk is an audio and web video conferencing provider. We've really come to embrace WebRTC as the platform for our future of technology," explained Dan Cunningham, CTO of ReadyTalk, in this SYS-CON.tv interview at WebRTC Summit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, Sandy Ca...