Java IoT Authors: Elizabeth White, Yeshim Deniz, Zakia Bouachraoui, Liz McMillan, William Schmarzo

Related Topics: @CloudExpo, Java IoT, Containers Expo Blog

@CloudExpo: Blog Post

Predictable Performance & Granular Control By @KellyMurphyGS | @CloudExpo [#Cloud]

In a highly virtualized environment, delivering predictable performance to VMs from a shared elastic resource is critical

Design Goal 2: Predictable Performance and Granular Control per VM

Traditional storage was designed to present a LUN to a server. This worked well until that server became a hypervisor running 10, 20 or 30 virtual servers, all pointing to the same LUN. Adding to the complexity, all hypervisors need to point to the same LUN to enable VM mobility or high availability. Architecturally, this creates a many-to-one relationship that makes optimization impossible. Each VM has a different I/O pattern. When they all mix together, the aggregate I/O becomes highly random, creating performance issues with no way to isolate problems or control I/O per VM.

Isolate and Optimize I/O per VM
The core component of the Gridstore architecture is the vController that operates in the host kernel and appears as a local SCSI device. From this privileged position, it isolates I/O from each of the VMs into independent swim lanes or channels from the hypervisor across the network and onto storage nodes in the Grid. This re-creates the1:1 relationship between VM and storage. Once I/O is isolated, each channel can be independently optimized according to what the VM is trying to achieve. Optimizations occur dynamically optimizing the I/O pattern to the demands of the VM.

System Center
All-Flash Hyper-Converged Infrastructure
Predictable Performance per VM
With the I/O isolated into channels per VM, the vController can now overlay QoS policies or set specific IOPS minimum/maximum per VM to give predictable performance per VM. Upon provisioning VMs through Virtual Machine Manager (VMM) and classifying them as a platinum service, Gridstore will deliver a platinum storage service. Furthermore, administrators can dial in a specific amount of IOPS to a given VM to ensure that it allocates a minimum amount of I/O to enable it to run optimally. Equally, lower-grade classes can be created or limits on IOPS can be dialed in to ensure that no VM consumes more than it requires. This is beneficial for service providers who want to offer different levels of service to customers, since it ensures they can deliver what they promise, resulting in predictable environments that run as desired for all workloads.

End-to-End QoS Architecture
Due to the unpredictable performance in virtual environments, QoS has become a hot feature. However not all QoS is the same. Three models for QoS are:

  1. Hypervisor based
  2. Array based
  3. End-to-end

A. Hypervisor- (Host-) Based QoS
Hypervisors create QoS in the hypervisor by throttling I/O. This is effective to ensure that one or more VMs do not consume all the IOPS, the noisy neighbor problem. This model, however, only operates in a single hypervisor host. It does not provide a global view (i.e., storage resources being consumed by other hosts, like a database on a physical server). Furthermore, a hypervisor-based model is not in control of the actual storage resource, making it impossible to allocate more resources to a particular VM or physical host to meet a minimum IOPS reserve.

B. Array-Based QoS
This is the opposite of the hypervisor model. In this configuration a storage array presents multiple LUNs to the hypervisor. Each LUN has a QoS priority attached to it, and I/O is more or less "allocated" to these LUNs based on the prioritization. Problems with this model include:

  • Being LUN based, it is not granular to the VM level
  • The I/O blender is still blending, which means I/O that passes through a LUN is blended together from multiple VMs, resulting in no per-VM granularity. With this, hundreds of VMs on that LUN compete for resources. When a VM on the highest priority LUN runs wild and consumes all resources, every other VM is impacted. According to the QoS policy, it did its job. It served the highest-priority LUN and starved the other LUNs. However, it does not know that all this I/O is coming from a single VM at the expense of others. It does not know that they are on the same highest-priority LUN
  • An array-based QoS model does not have any ability to control the flow from the hypervisor end. Rather, its domain of control is within the array.

C. End-to-End Per-VM I/O Control
In a highly virtualized environment, delivering predictable performance to VMs from a shared elastic resource is critical. Gridstore's vController technology isolates I/O from each VM at the source and enables end-to- end per-VM I/O control through the entire storage stack. By controlling both the resource consumption (VM hosts) and the resource allocation (storage resources from Grid nodes), Gridstore can precisely monitor demand as well as control the allocation of resources across the hypervisor hosts and storage resources. End-to-end VM I/O control enables the ability to dial in specific I/O per VM or class of VMs, thus ensuring SLAs are met. The result: complete control of resource allocation and utilization.

The post Design Goal 2: Predictable Performance and Granular Control per VM appeared first on Gridstore.

More Stories By Kelly Murphy

As a serial entrepreneur with a track record of bringing disruptive technologies to market, Kelly Murphy brings 15 years CEO experience with disruptive venture backed software companies. In 1998, almost a decade before the cloud became popular, Murphy founded Marrakech, the first software company that offered on-demand procurement and supply chain systems to over 30,000 trading partners including some of the world’s largest retailers, consumer food producers, packaging companies and utilities. After selling Marrakech in 2007, he turned his sights onto what was his largest obstacle in growing his previous business — storage.

In 2009, Murphy founded Gridstore — a pioneer of software-defined storage that is set to disrupt the traditional storage industry. Currently, he serves on Gridstore’s Board of Directors and is also the Chief Technology Officer. Originally from Canada, Murphy obtained his BS in Computer Science from Michigan Technological University, played Division I hockey and was the seventh pick of the New York Islanders in the 1984 entry draft.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments t...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Bill Schmarzo, Tech Chair of "Big Data | Analytics" of upcoming CloudEXPO | DXWorldEXPO New York (November 12-13, 2018, New York City) today announced the outline and schedule of the track. "The track has been designed in experience/degree order," said Schmarzo. "So, that folks who attend the entire track can leave the conference with some of the skills necessary to get their work done when they get back to their offices. It actually ties back to some work that I'm doing at the University of San...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...