Welcome!

Java IoT Authors: Liz McMillan, Elizabeth White, Pat Romanski, Yeshim Deniz, Frank Lupo

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, Containers Expo Blog, @CloudExpo

@DevOpsSummit: Blog Feed Post

Provisioning the Entire Stack | @DevOpsSummit #DevOps #Microservices

We’re currently in the process of stringing together applications at the various layers to do server provisioning

As we move into the world of complete data center automation, there is a whole new selection of issues that we are learning to resolve – from custom hardware to a variety of provisioning tools at each level of automation. These are not unexpected issues, but they certainly provide us with plenty to do while we’re trying to reduce the amount of busy work we have to do.

We’re currently in the process of stringing together applications at the various layers to do server provisioning, application provisioning, and (for internal apps at least) application deployment. The options out there are a pretty broad swath that runs from all-in-one solutions like Puppet with Razor to specialized solutions hooked together to run the entire automation process. The end result is the same, the differences are in implementation, vendor lock-in, and whether a full solution that can be implemented all at once is more important to an organization than a solution that has the most suited parts that can be implemented in parallel but have to be tied together.

Image complements of University Corporation for Atmospheric Research.

For those not familiar with myself or my employer StackIQ, the solutions available from my company run the gamut from specialized open source best-of-breed server provisioning (Stacki) to end-to-end Hadoop deployment. Which a given organization uses just depends upon which solution and what the organization’s relationship with StackIQ is.

As such, I am comfortable discussing the strengths and weaknesses of both styles of automation. After all, I’ve worked with both extensively, and can see the use cases that drive either one. There is no shortage of vendors in this space, and I will even mention a couple other than StackIQ, but my examples and use-cases will come from the various editions of Stacki, because I have worked on them more with that platform than others.

At the bottom level, pure-play server provisioning tools aim for best-of-breed usefulness. Open Source Stacki, for example, is focused on installing RHEL and CentoOS on target servers and configuring them to begin use. Of course few of the solutions on the market stop there, though a couple do. Stacki, for example, can be extended to install and configure agents from most of the application provisioning tools on the market. This means that server provisioning can hand off to application provisioning to achieve the full automation solution. It can also be extended to install specific applications as part of the OS install process. So a give server or set of servers can be configured to run Tomcat via installation, so that when Stacki finishes, the application server is installed and configured.

The drawback to this entry level is, of course, that such extensions must be developed by the user. While it is certainly possible to share extensions between users (for most tools anyway), and solicit feedback on the extensions being developed, there will still be work setting it up to work correctly. Some organizations – particularly those seeking a significant amount of control in their installations – find this to be an acceptable trade-off. Others would rather some of this foot work be done for them, or at least that they had ready access to people who specialize in this sort of thing.

That’s where the middle level comes in. This solution offers the functionality of the entry level, but adds either pre-built packages to install specific applications (like puppet agents or Tomcat) that can be easily marked as part of the install for a server or group of servers, or alternatively they offer the help of experts in setting these installations up – because once they’ve been configured, they (1) Don’t generally need to change much except to upgrade when new releases of the app come out and (2) Serve as examples that can be used to make other application customizations.

The issues with this solution are that it still takes time to get everything configured the first time, and it still requires a relatively intimate knowledge of the applications being deployed. For some shops these are not issues at all, for others they are show-stoppers that require significantly more investment than the organization is willing or able to make. That’s why there is the third level of integration for automation tools.

For some solutions in some organizations, the best answer is “push a button, get fully functional deployed servers running the software chosen.” A good example of this is big data. Deploying Hadoop by hand can be a daunting experience, yet timelines for Hadoop deployment far too often are tight. The third tier of integration is the system that knows what to do, and can juggle all the disparate pieces – hardware config, OS install, application install, configuration – at once. Push a button, reboot the machines intended for the Hadoop cluster, and a short time later there is a running Hadoop instance. I’m using Hadoop as an example here because of the glaring difference between when I’ve deployed it by hand and by automated provisioning (weeks to hours), but complex n-tier architectures stand to gain as much.

With these tools, an organization gets support built-in for a given set of applications, or gets the plug-ins to support a given application and adds them to one of the lower tiers. Then it is a simple case of defining (through command line, APIs, or spreadsheets) how the complex system should be deployed across target servers, and hitting a button. In a short time, the system is up and running with the target servers fully installed and doing their part. The issues with this solution are that there is a limit to the number of complex applications that have pre-packaged solutions, and of course, no internally developed application does. Professional services can help with this problem, but that adds expense and timeline to the project in a manner similar to the second tier of integration.

The best solution to these issues should be able to support all of the above integration tiers, but it should also be even more adaptable. There are several very good application provisioning tools on the market – Puppet, Chef, Saltstack, Ansible to name a few. A server provisioning tool, as the first step that runs in server-up datacenter automation, should be able to work reliably with all of these. Whether it enables users to hook up to them, or actually deploys the agents itself, the end result should be that a deployed server should be able to use the application provisioning that is currently in use at a given organization, and be able to change if that provisioning tool is replaced. Deeply integrated tools will work well with a single application provisioning tool, but tend toward vendor lock-in, because that tool is then harder to replace without replacing how servers are provisioned (which is no small chore in and of itself).

So when speccing out a datacenter automation or DevOps automation project, look for a server provisioning tool that can hand off to the application provisioning tool of choice, has some subset of application provisioning for complex systems (unless your application provisioning tool is already taking care of that of course), and can be fully automated. Partial automation is difficult at best, but it is the state of this growing market that some functionality is still not completely implemented in some products. It is easy enough to poke at products and determine if they suit the needs of your level of automation. A good example is that only about half of the products on the market make any attempt at RAID configuration as part of server provisioning. But if operators have to sit and configure RAID before an automated installation, then it’s not really fully automated, is it?

And of course I recommend that you consider automation. Partially because my employer plays in the space, but moreso because there are far more impactful things operations could be doing than sitting and watching an OS install, or typing in IP addresses, masks, and gateways.

More Stories By Don MacVittie

Don MacVittie is founder of Ingrained Technology, A technical advocacy and software development consultancy. He has experience in application development, architecture, infrastructure, technical writing,DevOps, and IT management. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.

@ThingsExpo Stories
As businesses evolve, they need technology that is simple to help them succeed today and flexible enough to help them build for tomorrow. Chrome is fit for the workplace of the future — providing a secure, consistent user experience across a range of devices that can be used anywhere. In her session at 21st Cloud Expo, Vidya Nagarajan, a Senior Product Manager at Google, will take a look at various options as to how ChromeOS can be leveraged to interact with people on the devices, and formats th...
SYS-CON Events announced today that Yuasa System will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Yuasa System is introducing a multi-purpose endurance testing system for flexible displays, OLED devices, flexible substrates, flat cables, and films in smartphones, wearables, automobiles, and healthcare.
Organizations do not need a Big Data strategy; they need a business strategy that incorporates Big Data. Most organizations lack a road map for using Big Data to optimize key business processes, deliver a differentiated customer experience, or uncover new business opportunities. They do not understand what’s possible with respect to integrating Big Data into the business model.
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they b...
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...