Click here to close now.

Welcome!

Java Authors: Carmen Gonzalez, Pat Romanski, Liz McMillan, Dan Ristic, Elizabeth White

Related Topics: Cloud Expo, Java, Microservices Journal, Linux, Virtualization, Big Data Journal

Cloud Expo: Article

Take Control of Your Schemalessness with Dynamic Schemas

Addressing the inflexibility of structured data by enabling schemaless data to be dynamically and logically structured

Static data structures have been at the heart of data processing tools since the dawn of computing, but they have always limited the flexibility of the organization leveraging the data. Recently, the rise of flexible formats like JSON have led to schemaless data as an attempt to increase agility. However, schemaless data have proven difficult to work with, because of hidden rigid structure in the form of implied schemas.

EnterpriseWeb addresses the problems of both the inflexibility of structured data as well as the impracticality of schemaless data, by enabling schemaless data to be dynamically and logically structured.

From the fixed-length fields of the 1950s, to the relational structures of modern database management systems, to the semistructured data formats XML and JSON, the structure of our data has always informed code about how it should be processed. Data are defined by their relationships, and we used to hard-code those relationships into rigid structures. That approach allows only one static view, which is difficult to work with, and even more difficult to change. Nevertheless, such rigid data structures - and the models that represent them - are an integral part of enterprise information management.

Traditional relational database management systems (RDBMSs) exemplify this point with their static entity-relationship models (ERMs) and tightly interconnected data structures. XML improves this situation slightly, allowing semi-structured information, but schemas still constrain flexibility and performance. With both approaches, fixed definitions, views, and reports limit the ability for businesses to freely transform information into insight and become obstacles to systemwide change.

The Rise of Schemalessness
This challenge of inflexible data structures has given rise to schemaless data. With JSON in particular, we can create whatever data structure we like when we author data. We don't have to shoehorn data into rigid data structures, thus allowing every record to have its own structure.

But there is a problem with schemaless data. Consider this simple task: how do you create a query for all the addresses in a particular Zip Code if every record has a different name or format for Zip Code? Schemalessness, after all, isn't magic - even schemaless data require some kind of metadata so the code will know how to process such information, what software development guru Martin Fowler calls an implied schema.

Implied schemas represent the structure inherent in any data record. If each address record has its own format, then that format provides the implied schema for that record. Dealing with implied schemas thus falls to the developer, who must figure out how to code software to process these implied schemas, which are different for each and every record.

In Fowler's tutorial on schemalessness, he explains the pros and cons of implied schemas. Despite acknowledging the power of schemalessness to support more flexible and responsive user experiences, he recommends avoiding it and implied schemas for developer convenience. Good advice with respect to traditional software, but the world of data is changing. Today we live in an increasingly schemaless world, where more often than not, the structure of our data is fluid or nonexistent.

Raising the Discussion to Dynamic Schemas
Fowler makes it clear that in the past it has been impractical from the developer's perspective to work systematically with schemaless data, because implied schemas are difficult to deal with. After all, structure is itself useful, and isn't the problem per se. Rather, how to avoid the limitations of static structure without falling into the trap of unmanageable schemaless data that is the real challenge.

EnterpriseWeb's unique approach to modeling solves this critically important challenge by leveraging dynamic schemas that have flexible, metadata-driven relationships with underlying information. Using metadata this way separates concerns, letting people consider relationships from multiple perspectives, rather than from a single static point of view. In addition, it's now possible to change and extend metadata to meet diverse business needs without disruption.

Instead of settling for complex ERMs with their inflexible, tightly coupled data structures or dealing with the coding complexities of implied schemas, developers can project dynamic schemas from the metadata simply by writing different transformations. As a result, dynamic schemas are developer friendly and dynamic - a welcome change from the difficult problem of schemalessness.

Add an Agent for Performance
So far so good, but how do we build software to process all such data in a general way, freeing ourselves from custom coding for implicit schemas? The solution is an intelligent agent.

EnterpriseWeb's intelligent agent, SmartAlex™, is a distributable transaction manager that resolves dynamic schemas for each interaction. Every human or system client interaction is a request for SmartAlex to interpret dynamic schemas (as well as other models and additional metadata) and translate them to a context-specific set of resources in order to construct a custom response.

This Agent-Oriented approach maximizes performance for such dynamic computing. In the background, SmartAlex handles all run time connection and transformation details, sparing programmers from manually integrating resources for varied and unanticipated uses, greatly improving IT productivity while enabling business agility.

SmartAlex logs all system events, indexes all new and updated resources, and tags all changes in relationships for detailed and navigable audit history. This practice creates a feedback loop as SmartAlex leverages the same indexed logs to guide its execution. Data, code, and user interface components, as well as connectors for federated services, systems, databases, and devices, can be updated or replaced without breaking related apps and processes - as SmartAlex is ‘aware' of the changes. In this way EnterpriseWeb supports real time exception and change management for resilient solutions that can evolve naturally.

The EnterpriseWeb Take
Schemalessness was a reaction to the limitations of structured data. People struggled with the constraints of static structure, and figured that if they simply got rid of structure, then the problem would go away. But this move was merely a shell game, as the limitations of fixed schemas shifted to implied schemas, now without the benefits of structure to inform the code responsible for their processing.

The solution is to raise the level of abstraction, and instead of arguing over fixed vs. implied schemas, to work at the dynamic schema level. Such an approach is model-driven, allowing application designers to build models that capture their data structures, and allowing an intelligent agent to use the metadata each model represents to meet the specific needs of each interaction. The real lesson here is that the solution to resolving the challenge of schemalessness combines both dynamic schemas and the action of the agent. Stay tuned to my next newsletter for more information.

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@ThingsExpo Stories
SYS-CON Events announced today that Alert Logic, the leading provider of Security-as-a-Service solutions for the cloud, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® and DevOps Summit 2015 New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo® and DevOps Summit 2015 Silicon Valley, which will take place November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, deploy, and manage applications integrating voice, video and data. He is the co-founder of TeleStax, a...
SYS-CON Events announced today that Akana, formerly SOA Software, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. Akana’s comprehensive suite of API Management, API Security, Integrated SOA Governance, and Cloud Integration solutions helps businesses accelerate digital transformation by securely extending their reach across multiple channels – mobile, cloud and Internet of Things. Akana enables enterprises to share data as APIs, connect and integrate applications, drive part...
VoxImplant has announced full WebRTC support in the newest versions of its Android SDK and iOS SDK. The updated SDKs, which enable audio and video calls on mobile devices, are now compatible with the WebRTC standard to allow any mobile app to communicate with WebRTC-enabled browsers, including Google Chrome, Mozilla Firefox, Opera, and, when available, Microsoft Spartan. The WebRTC-updated SDKs represent VoxImplant's continued leadership in simplifying the development of real-time communications (RTC) services for app developers. VoxImplant (built by Zingaya, the real-time communication servi...
SYS-CON Events announced today that CommVault has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. A singular vision – a belief in a better way to address current and future data management needs – guides CommVault in the development of Singular Information Management® solutions for high-performance data protection, universal availability and sim...
SYS-CON Events announced today that SafeLogic has been named “Bag Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. SafeLogic provides security products for applications in mobile and server/appliance environments. SafeLogic’s flagship product CryptoComply is a FIPS 140-2 validated cryptographic engine designed to secure data on servers, workstations, appliances, mobile devices, and in the Cloud.
SYS-CON Events announced today that StorPool Storage will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. StorPool is distributed storage software that allows service providers, enterprises and other cloud builders to run data storage on standard x86 servers, instead of using expensive and inefficient storage arrays (SAN).
The IoT Bootcamp is coming to Cloud Expo | @ThingsExpo on June 9-10 at the Javits Center in New York. Instructor. Registration is now available at http://iotbootcamp.sys-con.com/ Instructor Janakiram MSV previously taught the famously successful Multi-Cloud Bootcamp at Cloud Expo | @ThingsExpo in November in Santa Clara. Now he is expanding the focus to Janakiram is the founder and CTO of Get Cloud Ready Consulting, a niche Cloud Migration and Cloud Operations firm that recently got acquired by Aditi Technologies. He is a Microsoft Regional Director for Hyderabad, India, and one of the f...
SYS-CON Events announced today that kintone has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. kintone promotes cloud-based workgroup productivity, transparency and profitability with a seamless collaboration space, build your own business application (BYOA) platform, and workflow automation system.
SYS-CON Events announced today that Site24x7, the cloud infrastructure monitoring service, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Site24x7 is a cloud infrastructure monitoring service that helps monitor the uptime and performance of websites, online applications, servers, mobile websites and custom APIs. The monitoring is done from 50+ locations across the world and from various wireless carriers, thus providing a global perspective of the end-user experience. Site24x7 supports monitoring H...
SYS-CON Events announced today that Intelligent Systems Services will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Established in 1994, Intelligent Systems Services Inc. is located near Washington, DC, with representatives and partners nationwide. ISS’s well-established track record is based on the continuous pursuit of excellence in designing, implementing and supporting nationwide clients’ mission-critical systems. ISS has completed many successful projects in Healthcare, Commercial, Manufacturing, ...
WebRTC is an up-and-coming standard that enables real-time voice and video to be directly embedded into browsers making the browser a primary user interface for communications and collaboration. WebRTC runs in a number of browsers today and is currently supported in over a billion installed browsers globally, across a range of platform OS and devices. Today, organizations that choose to deploy WebRTC applications and use a host machine that supports audio through USB or Bluetooth can use Plantronics products to connect and transit or receive the audio associated with the WebRTC session.
The best mobile applications are augmented by dedicated servers, the Internet and Cloud services. Mobile developers should focus on one thing: writing the next socially disruptive viral app. Thanks to the cloud, they can focus on the overall solution, not the underlying plumbing. From iOS to Android and Windows, developers can leverage cloud services to create a common cross-platform backend to persist user settings, app data, broadcast notifications, run jobs, etc. This session provides a high level technical overview of many cloud services available to mobile app developers, includi...
SYS-CON Events announced today that B2Cloud, a provider of enterprise resource planning software, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. B2cloud develops the software you need. They have the ideal tools to help you work with your clients. B2Cloud’s main solutions include AGIS – ERP, CLOHC, AGIS – Invoice, and IZUM
The WebRTC Summit 2015 New York, to be held June 9-11, 2015, at the Javits Center in New York, NY, announces that its Call for Papers is open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 16th International Cloud Expo, @ThingsExpo, Big Data Expo, and DevOps Summit.
SYS-CON Events announced today that Tufin, the market-leading provider of Security Policy Orchestration Solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. As the market leader of Security Policy Orchestration, Tufin automates and accelerates network configuration changes while maintaining security and compliance. Tufin's award-winning Orchestration Suite™ gives IT organizations the power and agility to enforce security policy across complex, multi-vendor enterprise networks. With more than 1...
As enterprises move to all-IP networks and cloud-based applications, communications service providers (CSPs) – facing increased competition from over-the-top providers delivering content via the Internet and independently of CSPs – must be able to offer seamless cloud-based communication and collaboration solutions that can scale for small, midsize, and large enterprises, as well as public sector organizations, in order to keep and grow market share. The latest version of Oracle Communications Unified Communications Suite gives CSPs the capability to do just that. In addition, its integration ...
SYS-CON Events announced today that Cloudian, Inc., the leading provider of hybrid cloud storage solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cloudian, Inc., is a Foster City, California - based software company specializing in cloud storage software. The main product is Cloudian, an Amazon S3-compliant cloud object storage platform, the bedrock of cloud computing systems, that enables cloud service providers and enterprises to build reliable, affordable and scalable cloud storage solu...
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...