Welcome!

Java IoT Authors: Liz McMillan, Elizabeth White, Pat Romanski, Yeshim Deniz, Paul Simmons

Related Topics: @DXWorldExpo, @CloudExpo, Apache

@DXWorldExpo: Article

Analytics in Decision-Making Workflow | @CloudExpo #BigData #Microservices

Big Data shouldn’t be restricted to data scientists

Putting Analytics into the Decision-Making Workflow with Apache Spark

Data-driven businesses use analytics to inform and support their decisions. In many companies, marketing, sales, finance, and operations departments tend to be the earliest adopters of data analytics, with the rest of the business lagging behind. The goal for many organizations now is to make analytics a natural part of most-if not every-employee's daily workflow. Achieving that objective typically requires a shift in the corporate culture, and ready access to user-friendly data analytics tools.

Big Data Shouldn't Be Restricted to Data Scientists
Big Data experts, when discussing the process of integrating data analysis into the workflow across an enterprise, often talk blithely about how users can easily leverage their SQL skills to query data. The problem is that not everyone has SQL skills-or even knows what SQL is.

Companies who plan to transform themselves into data-driven, lean businesses may want to consider the fact that every employee really doesn't need to be a data scientist. Focus the majority of training efforts (including how to run basic SQL queries, if necessary) on the employees whose jobs involve fact-based decision-making.

Making employees wait for IT to manage schemas and setup ETL tasks is counter-productive. In a busy company, by the time data is prepped for analysis, it may have lost some of its actionable relevance. Instead, provide robust self-service data analysis tools, such as Apache Drill, to enable users to extract the most value possible from data stored in Hadoop. This frees employees to work with data in native formats-schema-less data, nested data, and data with rapidly-evolving schemas-with limited to no IT involvement.

Self-service data tools also enable explorative queries. Users can explore the data directly and extend their analysis effortlessly, with no need to wait for IT to prep additional data sets. Analysis can then extend past known, structured data, to semi-structured and unstructured data, such as call center logs, videos, spreadsheets, social media data, clickstream data, web log files, and external data (such as publicly available industry data)-allowing a business to gain big picture, actionable insights on the fly.

Apache Spark: Bringing New Efficiencies to Big Data Analysis
Agile companies that rely on data analysis performed in near-time and real-time also need solutions that can rapidly process large data sets. Apache Spark, an in-memory data processing framework, is increasingly the solution of choice.

Spark is a framework providing parallel, distributed data processing. Spark can be deployed through Apache Hadoop via Yarn, Apache Mesos, or its own standalone cluster manager. It can serve as a foundation for other data processing frameworks, and supports programming languages including Scala, Java, and Python. Data can be accessed in HDFS, Cassandra, HBase, Hive, Tachyon, and any Hadoop data source.

Data sets can be pinned in memory with Spark, which boosts application performance noticeably. Spark also provides speed improvements for applications running on disk and enables MapReduce to support interactive queries and stream processing far more efficiently.

And Spark eliminates the need for separate, distributed systems to process, for example, batch applications, interactive queries, iterative algorithms, and/or streaming. With Spark, all of these processing types are supported by the same engine, reducing management chores and making the processes easier to combine.

Businesses can count on Spark's benefits over the long-term. Spark, initially conceived as a project at UC Berkeley in California, moved to the Apache Software Foundation in 2013 and became a top level project in 2014. Apache top level projects, which include Hadoop, Spark, and httpd, is a designation that indicates a project has strong community backing from developers and users-and has proved its worth. More than 50 companies currently list themselves on Spark's "Powered By" page.

Putting Data-Driven Intelligence to Work
Big Data incarnates multiple processes-collection, cleansing, integration, management, governance, security, analysis, and decision-making-all of which need to be in place before a company can consider itself data-driven. Oddly, the decision-making process itself tends to get the least attention.

Gaining real ROI from a Big Data project requires more than fast tools and a solid plan to enable users to incorporate analysis-driven decision-making into their workflow. Quick discovery of exciting new insights in data has no benefit if a company doesn't have a process that enables an equally speedy and effective response to that new intelligence. When devising (or revising) your Big Data project, ensure that you build in an implementation process that enables analysis to be transformed into action.

And finally, a word of warning about real-time analysis: It's easy to lose sight of long-range goals when you're immersed in the moment. Ensure that business goals are aligned with data analysis activities, and establish KPIs to monitor the success of data-driven initiatives. Big Data should provide a company with a sustainable competitive edge.

To explore more of what Spark has to offer, jump over to Getting Started with Apache Spark: From Inception to Production, a free interactive ebook by James A. Scott.

More Stories By Jim Scott

Jim has held positions running Operations, Engineering, Architecture and QA teams in the Consumer Packaged Goods, Digital Advertising, Digital Mapping, Chemical and Pharmaceutical industries. Jim has built systems that handle more than 50 billion transactions per day and his work with high-throughput computing at Dow Chemical was a precursor to more standardized big data concepts like Hadoop.

@ThingsExpo Stories
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
"We view the cloud not as a specific technology but as a way of doing business and that way of doing business is transforming the way software, infrastructure and services are being delivered to business," explained Matthew Rosen, CEO and Director at Fusion, in this SYS-CON.tv interview at 18th Cloud Expo (http://www.CloudComputingExpo.com), held June 7-9 at the Javits Center in New York City, NY.
The Founder of NostaLab and a member of the Google Health Advisory Board, John is a unique combination of strategic thinker, marketer and entrepreneur. His career was built on the "science of advertising" combining strategy, creativity and marketing for industry-leading results. Combined with his ability to communicate complicated scientific concepts in a way that consumers and scientists alike can appreciate, John is a sought-after speaker for conferences on the forefront of healthcare science,...
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
In his session at Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to maximize project result...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Archi...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abilit...
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.