Welcome!

Java IoT Authors: Yeshim Deniz, Liz McMillan, Matt Davis, Ben Uher, Jyoti Bansal

Related Topics: SDN Journal, Java IoT, Linux Containers, Containers Expo Blog, @CloudExpo, Cloud Security

SDN Journal: Blog Post

Fabric Engineering Is More than Traffic Engineering

Traffic engineering has taken on a very specific meaning, very much related to the way traffic is mapped onto MPLS

It is human nature to try and relate new information and new ways of doing things to something that we know, something we are familiar with. Often when we talk about the way we fit traffic onto a Plexxi mesh network, the reaction is “I know what you mean, you are doing traffic engineering like we (used to) do in MPLS”. The response to that is usually “kinda, but not really”.

In the most basic meaning, everything that has to do with the placement of traffic onto links, routing and forwarding choices being programmed, etc., would be part of Traffic Engineering. But like too many words and phrases in our networking dictionary, traffic engineering has taken on a very specific meaning, very much related to the way traffic is mapped onto MPLS and like networks. For us, its a bit different. We build ethernet and IP networks that use packet by packet forwarding rules and while individual flows may have a hop by hop crafted path through the network, a very large portion of traffic travels using regular L2 and L3 forwarding tables. We just construct those tables differently than the typical network. Dare I say we engineer the tables to ensure forwarding occurs the way we calculated it to go.

Mike Bushong yesterday talked about equal cost paths and shortest path algorithms. We have discussed our views several times in this and other forums, and it is because we have a fundamental believe that  it can be done better. A Plexxi network consists of switches that are connected together using a variety of optical technologies with a mesh of 10GbE Ethernet connections. These point to point connections between switches for the basis of the L2 and L3 connectivity that exists, on top of which Plexxi Control Engineers the forwarding behavior of the fabric (skipping the part where Control can actually change those point to point connections for now).

So far this still smells like Traffic Engineering. I have a network, I have traffic I need to put on the network. The goal is simple, maximize the available network capacity. Use it as best as you can. All of it. While providing the best possible service for the providers and consumers of the traffic. But the way we do it is just a little different.

Our fabric is engineered based on a set of information sources. The most obvious of those sources is the actual network that is constructed. How our switches are connected together, and what initial L1/L2 topography (mesh of point to point networks) is created as part of our default connectivity. Our second source of information is the set of Affinities defined for the network. What application relationships are explicitly described, what is the requested network behavior for them and most importantly, where can I find these application components on the network. The latter is learned of course, not defined by the operator. The third component is actual traffic utilization on the fabric. What links are in use and how much of it.

These three inputs go into what we call the fitting engine. Heavy duty graph theory that we have blogged about before. The goal: resolve all Affinity constraints on top of the network infrastructure provided, and provide Non Equal Weighted Egress Based Multipath Trees for all non Affinitized traffic to ensure it gets the best service and spreads traffic as best as we can across all available paths between any two points in the fabric. That last one is a mouthful and not really what we call it, but at NFD7 yesterday I used it jokingly as the best descriptive term that articulates all it actually is. Last, a set of backup paths are calculated for link and switch failures. This can number 100s of paths easily. The results of these calculations are passed back to the switches in a multi phase commit fashion, ensuring that all switches will start using the same forwarding directions at the same time. Some of these are used in the switches as explicit flow like rules (similar to perhaps OpenFlow), others are used to populate forwarding tables when new MAC addresses are learned, or ARPs are resolved.

The result is a very carefully constructed description of connectivity from anywhere to anywhere on the fabric, satisfying the needs of Affinities, and balancing the remainder of the traffic across all available bandwidth. And once completed, the next set of traffic statistics that are entered into the overall traffic modeling will provide an even better tuned view of reality. When new Affinities are defined, you can run partial computations to just layer these new requirements on top of the previously calculated topologies.

Traffic Engineering has a very 90’s meaning attached to it. At least for me it does. Maybe this is Traffic Engineering 3.0 for the Datacenter and Data Center Interconnects (see last week’s blog post). Maybe the name does not matter. It’s a very mathematically engineered fabric. Mike said it well at NFD7: we are really a math company, masquerading as a software company, masquerading as a hardware company.

[Today's fun fact: Coconuts kill more people than sharks. Each year 150 people die from falling coconuts. Umbrellas make for safer shade than coconut trees.]

The post Fabric Engineering is more than Traffic Engineering appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

@ThingsExpo Stories
SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buyers...
SYS-CON Events announced today that Infranics will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Since 2000, Infranics has developed SysMaster Suite, which is required for the stable and efficient management of ICT infrastructure. The ICT management solution developed and provided by Infranics continues to add intelligence to the ICT infrastructure through the IMC (Infra Management Cycle) based on mathemat...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 20th Cloud Expo, which will take place on June 6-8, 2017 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 add...
SYS-CON Events announced today that SD Times | BZ Media has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. BZ Media LLC is a high-tech media company that produces technical conferences and expositions, and publishes a magazine, newsletters and websites in the software development, SharePoint, mobile development and commercial UAV markets.
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloudistics delivers a complete public cloud experience with composable on-premises infrastructures to medium and large enterprises. Its software-defined technology natively converges network, storage, compute, virtualization, and management into a ...
Now that the world has connected “things,” we need to build these devices as truly intelligent in order to create instantaneous and precise results. This means you have to do as much of the processing at the point of entry as you can: at the edge. The killer use cases for IoT are becoming manifest through AI engines on edge devices. An autonomous car has this dual edge/cloud analytics model, producing precise, real-time results. In his session at @ThingsExpo, John Crupi, Vice President and Eng...
There are 66 million network cameras capturing terabytes of data. How did factories in Japan improve physical security at the facilities and improve employee productivity? Edge Computing reduces possible kilobytes of data collected per second to only a few kilobytes of data transmitted to the public cloud every day. Data is aggregated and analyzed close to sensors so only intelligent results need to be transmitted to the cloud. Non-essential data is recycled to optimize storage.
"I think that everyone recognizes that for IoT to really realize its full potential and value that it is about creating ecosystems and marketplaces and that no single vendor is able to support what is required," explained Esmeralda Swartz, VP, Marketing Enterprise and Cloud at Ericsson, in this SYS-CON.tv interview at @ThingsExpo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
As businesses adopt functionalities in cloud computing, it’s imperative that IT operations consistently ensure cloud systems work correctly – all of the time, and to their best capabilities. In his session at @BigDataExpo, Bernd Harzog, CEO and founder of OpsDataStore, will present an industry answer to the common question, “Are you running IT operations as efficiently and as cost effectively as you need to?” He will expound on the industry issues he frequently came up against as an analyst, and...
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" ...
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor - all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex softw...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, will discuss the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information,
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Apache Hadoop is emerging as a distributed platform for handling large and fast incoming streams of data. Predictive maintenance, supply chain optimization, and Internet-of-Things analysis are examples where Hadoop provides the scalable storage, processing, and analytics platform to gain meaningful insights from granular data that is typically only valuable from a large-scale, aggregate view. One architecture useful for capturing and analyzing streaming data is the Lambda Architecture, represent...