Java IoT Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Roger Strukhoff, Pat Romanski

Related Topics: @DevOpsSummit

@DevOpsSummit: Blog Feed Post

Continuous Delivery, Real-Time Test Analysis By @XebiaLabs | @DevOpsSummit [#DevOps]

You have to plan your Continuous Delivery pipeline with quality in mind from the outset

Why Continuous Delivery is Nothing Without Real-Time Test Analysis

By Victor Clerc

TL;DR: There is no point shipping fast if everything is broken!

Pushing frequent releases of high quality software to customers is beneficial for everyone. There’s a great deal of business value in delivering software to the market faster than your competitors. Once realized, ideas can be rejected or pursued quickly. Customers can see their feedback shaping the product. Flaws, defects, and omissions can be addressed without delay.

But, setting up a Continuous Delivery pipeline is about more than speed. How do you ensure that things don’t start breaking all over the place?

It’s an established fact that the later we encounter a defect or a problem in software development, the more expensive and difficult it is to fix. Accurately measuring quality and building it into the pipeline may seem like a lot of upfront effort, but it will pay dividends quickly.

Step 1: Build solid foundations

You have to plan your Continuous Delivery pipeline with quality in mind from the outset. The only way to effectively do that is to design tests before development really begins, to continually collect metrics, and to build a test automation architecture integrated into your Continuous Delivery pipeline. The test automation architecture defines the setup of test tools and tests throughout the pipeline and should support flexibility and adaptability to help you meet your business objectives.

This test automation architecture should facilitate easy selection of just the right tests to be performed to match the flow of your software through your pipeline as it gets closer to release. From unit tests and code level security, through functional and component testing, to integration and performance testing. The further the software progresses along the pipeline, the greater the number of dependencies on other systems, data, and infrastructure components – and, the more difficult it is to harmonize these variables. Making sane selections of tests to run at each moment in the pipeline becomes more and more important in order to focus on the area of interest.

Step 2: Fail fast

If we wait until functionality has been developed to write automated functional tests, we are creating a bottleneck. Designing tests during or at the end of a sprint is delaying feedback that we need right now. Why not apply the same test-driven development mindset we employ for unit tests to functionality? So, design all tests as soon as possible: refining the backlog for subsequent sprints should include discussing and designing tests.

The development team is greatly helped when for each piece of functionality a set of examples of expected system behavior is available. These examples, with some syntactic guidelines and using readily available tools, can serve as automated acceptance tests. Starting with failing tests and working to eliminate them maintains a laser focus on what developers should be doing. For this to work well, all stakeholders must get together at the beginning to design the acceptance tests.

The alternative is a drip-feed of failure as new stakeholders climb on board with fresh concerns as the software gets closer to release. The later a failure occurs, the more disruptive it is, and the more difficult it is to do anything to alleviate the relevant stakeholder’s concerns. Pull everyone in at the start, design the tests together, run the most important tests first, and start collecting the feedback you need about software quality from day one.

Step 3: Automate and optimize the process

A concrete set of acceptance criteria in the form of automated tests gives you a clear signal about the health of your software. The easier it is to run these tests, the better, which is why you should automate where possible. There will be some intangibles that will prove tough to automate, but you may need them for a true measure of quality. Not all acceptance tests can or should be automated, but automating functional tests makes a great deal of sense.

It will be necessary from time to time to add new example test cases in to augment the existing functionality. It will also be necessary to optimize the tests available in the regression set to prevent this set from growing and containing superfluous tests.

You can’t test everything all the time. Be pragmatic when you define which tests to run in any given situation. You need flexibility to select the right tests for inclusion and exclusion according to a variety of quality attributes.

Step 4: Maintain a real-time quality overview

If a test is going to fail, it’s better to know about it now. All the stakeholders should know about failures as soon as possible. If the software is passing your tests and meeting the acceptance criteria then you can feel confident about the overall quality and proceed with the release.

Keeping things specific with examples and creating these acceptance tests as early as possible will increase the efficiency of your software development process. Furthermore, when possible reasons for failing tests can be identified through adequate monitoring, this may help to correct defects as soon as possible. The predictive value of trend analysis can also help you to take adequate measures before quality thresholds, such as a maximum duration for test execution, are breached.

Automated continuous qualification of all relevant test results (across all relevant test tools) is what enables you to provide real-time quality feedback. With this real-time quality feedback built into your Continuous Delivery pipeline, you can ensure that your standards never slip.

Join the XL Test beta programme!

We have built XL Test to allow you to achieve these steps. If you’re interested and would like to join our beta programme, please drop us an email. Start making sense of all your test data and get real-time quality feedback built into your Continuous Delivery pipeline today!

The post Why Continuous Delivery is Nothing Without Real-Time Test Analysis appeared first on XebiaLabs.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

@ThingsExpo Stories
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect at GE, and Ibrahim Gokcen, who leads GE's advanced IoT analytics, focused on the Internet of Things / Industrial Internet and how to make it operational for business end-users. Learn about the challenges posed by machine and sensor data and how to marry it with enterprise data. They also discussed the tips and tricks to provide the Industrial Internet as an end-user consumable service using Big Data Analytics and Industrial C...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
With privacy often voiced as the primary concern when using cloud based services, SyncriBox was designed to ensure that the software remains completely under the customer's control. Having both the source and destination files remain under the user?s control, there are no privacy or security issues. Since files are synchronized using Syncrify Server, no third party ever sees these files.
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Rodrigo Coutinho is part of OutSystems' founders' team and currently the Head of Product Design. He provides a cross-functional role where he supports Product Management in defining the positioning and direction of the Agile Platform, while at the same time promoting model-based development and new techniques to deliver applications in the cloud.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
delaPlex is a global technology and software development solutions and consulting provider, deeply committed to helping companies drive growth, revenue and marketplace value. Since 2008, delaPlex's objective has been to be a trusted advisor to its clients. By redefining the outsourcing industry's business model, the innovative delaPlex Agile Business Framework brings an unmatched alliance of industry experts, across industries and functional skillsets, to clients anywhere around the world.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...