Welcome!

Java IoT Authors: Pat Romanski, Liz McMillan, AppDynamics Blog, Dana Gardner, Automic Blog

Related Topics: Java IoT, Industrial IoT, Microservices Expo, Microsoft Cloud, Open Source Cloud, Containers Expo Blog

Java IoT: Blog Post

Efficiency in Development Workflows: Immutable Infrastructure

Immutable Infrastructure and Continuous Deployment work great together

After reading Chad Fowlers excellent blogpost about immutable deployments at 6Wunderkinder, we wanted to share our views on immutability in infrastructure.

Our definition of Immutable Infrastructure:

  1. Automate the setup and deployment for every part and every layer of your infrastructure.
  2. Never change any part of your system once it is deployed. If you need to change it, deploy a new system.

For example, instead of deploying into an existing EC2 instance, start a new server, deploy there and point your load balancer to the new server. Then remove the old server.

Replacing a system at the lowest level you can forces you to automate every deployment step.

Immutable infrastructure and Continuous deployment work great together. Completely replacing, instead of updating, an existing part of your infrastructure makes your deployments less complex.

Test driven development, Continuous Deployment and Immutable Infrastructure are strategies we have been using on the Codeship for a long time.

For Immutable Infrastructure you need cloud servers and a virtualised environment.

Cloud servers are building blocks
In his AWS re:Invent Keynote Werner Vogels talked about Cloud servers as building blocks for larger systems. Jamie Begin wrote a great blog post on cloud serves as building blocks, based on the Keynote.

Today cloud instances are still used like physical hardware in the past. You set it up once and update it whenever necessary. The problem is that cloud servers are not meant to be reliable or durable.

Their advantage is that they are standardised and easy to replace. Cloud servers are like Lego pieces that can be changed whenever necessary. If you want to have a different color or the lego piece breaks, just put in a new one. You wouldn't repair a lego piece, would you?

Immutable Infrastructure is like building with lego blocks

You wouldn't repair a lego piece. Just grab a new one.

Our Experience with Immutable infrastructure
Our web application, the Mothership, is hosted on Heroku and has therefore always been immutable. Whenever we deploy a new version, Heroku builds the Slug and replaces current instances with it. We have enabled Herokus Zero Downtime support.

Our test server infrastructure, the Checkbot, is hosted on AWS since August 2012. Whenever we want to change the test servers, we build a completely new Amazon AMI, test it and replace the old machines with the new AMI. We will go into more detail about this in our next blogpost.

By replacing every part of our infrastructure, often several times a day, we feel very comfortable with releasing changes. This workflow allows us to improve our service very quickly.

Advantages of an Immutable Infrastructure
There are many more advantages to Immutable Infrastructure than the following, but we have found these to be the most important ones to us:

  • Going back to an old version is easy, as you have the old image available.
  • Every change to the infrastructure needs to be in a script. Any server can be removed at any time and will take manual changes with it.
  • It's easy to have a production-like system on development machines.
  • You have an incentive to speed up the time your servers need to be built. We will talk about this in future blogposts.
  • Setting up staging systems is easy and can be automated.
  • Testing the new infrastructure in isolation is possible.

Challenges with Immutable Infrastructure

Of course this approach also has its challenges. Especially around tooling.

  • Better and standardised tooling is necessary, although new tools like Packer make it easier.
  • Setting up automation for immutable infrastructure has higher costs at the beginning.
  • Fixing problems is slower as you can't just SSH into an existing server. It needs to be redeployed.
  • There needs to be a way to reliably replace a server without impacting the whole system. Queuing and proxies that can store requests for a while are helpful.
  • Replacing databases continuously is hard.

Conclusion
Fixing broken servers instead of replacing them is a waste of time. It slows down the development and deployment cycle.

Test-Driven Development, Continuous Deployment and Immutable Infrastructure are practices every team should use. Together these practices help build reliable and high quality software that can be changed at any time. Being able to go back to an old version of your system in seconds allows you to experiment and innovate at a much faster pace.

Over the last months different tools like Packer or Docker have been released that make Immutable Infrastructure a lot easier.

In our next blog post we will show you in detail how we deploy our testing infrastructure several times a day. In future blog posts we will introduce Packer, Docker and other tools and show you how to rebuild your infrastructure constantly. Stay tuned!

Further Info:

More Stories By Manuel Weiss

I am the cofounder of Codeship – a hosted Continuous Integration and Deployment platform for web applications. On the Codeship blog we love to write about Software Testing, Continuos Integration and Deployment. Also check out our weekly screencast series 'Testing Tuesday'!

@ThingsExpo Stories
The IoT has the potential to create a renaissance of manufacturing in the US and elsewhere. In his session at 18th Cloud Expo, Florent Solt, CTO and chief architect of Netvibes, discussed how the expected exponential increase in the amount of data that will be processed, transported, stored, and accessed means there will be a huge demand for smart technologies to deliver it. Florent Solt is the CTO and chief architect of Netvibes. Prior to joining Netvibes in 2007, he co-founded Rift Technologi...
Unless your company can spend a lot of money on new technology, re-engineering your environment and hiring a comprehensive cybersecurity team, you will most likely move to the cloud or seek external service partnerships. In his session at 18th Cloud Expo, Darren Guccione, CEO of Keeper Security, revealed what you need to know when it comes to encryption in the cloud.
We're entering the post-smartphone era, where wearable gadgets from watches and fitness bands to glasses and health aids will power the next technological revolution. With mass adoption of wearable devices comes a new data ecosystem that must be protected. Wearables open new pathways that facilitate the tracking, sharing and storing of consumers’ personal health, location and daily activity data. Consumers have some idea of the data these devices capture, but most don’t realize how revealing and...
What are the successful IoT innovations from emerging markets? What are the unique challenges and opportunities from these markets? How did the constraints in connectivity among others lead to groundbreaking insights? In her session at @ThingsExpo, Carmen Feliciano, a Principal at AMDG, will answer all these questions and share how you can apply IoT best practices and frameworks from the emerging markets to your own business.
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
The best-practices for building IoT applications with Go Code that attendees can use to build their own IoT applications. In his session at @ThingsExpo, Indraneel Mitra, Senior Solutions Architect & Technology Evangelist at Cognizant, provided valuable information and resources for both novice and experienced developers on how to get started with IoT and Golang in a day. He also provided information on how to use Intel Arduino Kit, Go Robotics API and AWS IoT stack to build an application tha...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, discussed the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filterin...
Is your aging software platform suffering from technical debt while the market changes and demands new solutions at a faster clip? It’s a bold move, but you might consider walking away from your core platform and starting fresh. ReadyTalk did exactly that. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue and over a decade of audio conferencing product development to start an innovati...
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
Early adopters of IoT viewed it mainly as a different term for machine-to-machine connectivity or M2M. This is understandable since a prerequisite for any IoT solution is the ability to collect and aggregate device data, which is most often presented in a dashboard. The problem is that viewing data in a dashboard requires a human to interpret the results and take manual action, which doesn’t scale to the needs of IoT.
What does it look like when you have access to cloud infrastructure and platform under the same roof? Let’s talk about the different layers of Technology as a Service: who cares, what runs where, and how does it all fit together. In his session at 18th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, an IBM company, spoke about the picture being painted by IBM Cloud and how the tools being crafted can help fill the gaps in your IT infrastructure.
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
"C2M is our digital transformation and IoT platform. We've had C2M on the market for almost three years now and it has a comprehensive set of functionalities that it brings to the market," explained Mahesh Ramu, Vice President, IoT Strategy and Operations at Plasma, in this SYS-CON.tv interview at @ThingsExpo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Traditional IT, great for stable systems of record, is struggling to cope with newer, agile systems of engagement requirements coming straight from the business. In his session at 18th Cloud Expo, William Morrish, General Manager of Product Sales at Interoute, outlined ways of exploiting new architectures to enable both systems and building them to support your existing platforms, with an eye for the future. Technologies such as Docker and the hyper-convergence of computing, networking and sto...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.