Welcome!

Java IoT Authors: Elizabeth White, Liz McMillan, William Schmarzo, SmartBear Blog, Charlotte Spencer-Smith

Blog Feed Post

Using the Python SDK for Monitis Custom Monitors

One advantage of using Monitis to monitor your systems and applications is the flexibility to use either the native agent or custom monitor code written in virtually any language. For custom monitors, the REST API provides the basic foundation to interface programmatically with Monitis. For many popular languages, there are open source SDKs available to make the process of interfacing with Monitis even easier. You can find links to Java, Perl, PHP, Ruby, C#, PowerShell, and VisualBasic SDKs and example scripts at http://monitis.com/api/api.html#sdk.

The Monitis Sandbox

For this article, we’ll be using the Monitis sandbox, a free development environment available to aid in the creation of Monitis custom monitors. You can sign up for an account at http://sandbox.monitis.com/.

Netstat

Netstat is a command available on unix systems to provide information about the state of the network interfaces. It has several modes, one of which can display statistics about the number of packets and datagrams sent and received. While there is no common API across unix systems to access this data, it is possible to extract this information from the text output of the program. Since the output varies slightly from system to system, it is necessary to take these variations into account when using that output.

Netstat examples using the Python Monitis SDK

The examples for this article are available at https://github.com/monitisexchange/Python-SDK/tree/master/examples. There are two examples. The first, monitis_netstat.py, can be used as is to create, list, update, and delete netstat monitors. The second, monitis_netstat_minimal.py, is a minimal implementation intended to illustrate the concepts for this article, but lacking the structure to handle command line arguments, different operating systems, etc.

monitis_netstat_minimal.py

Let’s take a look at monitis_netstat_minimal.py, line by line. It illustrates many of the core concepts in the Python SDK.

The following lines are imports from the Python standard library, used for calling netstat, searching its output, and comparing output from netstat with that retrieved via the SDK.

from subprocess import Popen, PIPE
from re import findall
from datetime import datetime

Next are the imports from the Monitis Python SDK itself, used for creating new monitors and fetching existing monitors.

from monitis.monitors.custom import CustomMonitor, get_monitors
from monitis.monitors.params import ResultParams, DataType
from monitis.api import Monitis

The Monitis.sandbox flag causes the SDK to use the Monitis developer’s sandbox, http://sandbox.monitis.com. By default, when this flag isn’t set, it uses the production environment, http://www.monitis.com.

Monitis.sandbox = True

The following block creates a new monitor. ResultParams encapsulates the format for the parameters that a monitor accepts. One or more of them can be passed in to the next line. CustomMonitor.addMonitor takes the parameters and creates a monitor based on them. It returns a new CustomMonitor object that represents the monitor in the Monitis API.

rp = ResultParams(
    'tcp_packets_in','TCP Packets In','pkts/s', DataType('integer'))
cm = CustomMonitor.add_monitor(rp, name='netstat monitor', tag='netstat')
print "Created monitor: %s" % cm.monitor_id

This code extracts the interesting data from the netstat output. In this case, that data is the count of TCP packets received. This is the only block of code in the minimal sample program that really isn’t dependent on the API. A different monitor could get data in any number of ways. Note that this call to netstat is based on OS X style output, but only a small change to the command and matching string is required to adapt it to virtually any unix-like operating system.

subproc = Popen(
    'netstat -s -p tcp | grep "packets received$"', shell=True, stdout=PIPE)
(netstat_out, netstat_err) = subproc.communicate()
count = findall('(\d+) packets received', netstat_out)[0]

Next, add_result takes the new data, and posts it in the format defined earlier in ResultParams.

cm.add_result(tcp_packets_in=count)

Here, we check that retrieving the results from the monitor gets back the same data that we put in.

today = datetime.utcnow()
result = cm.get_monitor_results(today.year, today.month, today.day)

Finally, delete the monitor. This exists in this example so that it cleans up after itself and doesn’t cause unused custom monitors to pile up.

cm.delete_monitor()

In typical usage, a monitor is only deleted when that data is no longer needed. This principle is explored further in monitis_netstat.py.

monitis_netstat.py

Under normal circumstances, a monitor is created once, data is updated on a periodic basis over time, and then that data is viewed in the Monitis web interface. To ilustrate this, monitis_netstat.py has command line flags to create, list, delete, and update a monitor.

Create

Create a new netstat monitor:

$ examples/monitis_netstat.py -s -c netstat_mon
Using name netstat_mon
Created new netstat monitor

List

List the monitors to verify that ours was created:

$ examples/monitis_netstat.py -s -l
2267	netstat_mon

Update

Run the monitor 10 times over five minutes to gather some data to display.

$ while (true) do examples/monitis_netstat.py -s -u 2267; sleep 30; done
1253222, 1616400, 381497, 551524
1253393, 1616514, 381505, 551536
1253436, 1616553, 381512, 551545
1253468, 1616580, 381514, 551549
1253485, 1616596, 381517, 551558
1253526, 1616633, 381522, 551609
1253564, 1616666, 381531, 551622
1253599, 1616695, 381536, 551629
1253623, 1616716, 381538, 551633
1253647, 1616740, 381544, 551638

Once this is complete, the results can be seen in the web interface.

Delete

$ examples/monitis_netstat.py -s -d 2267
Monitor 2267 deleted
$ examples/monitis_netstat.py -s -l

Try it

To get the code and try this yourself, you can clone a copy from GitHub, install the Monitis SDK module, and run it on your own system.

git clone git://github.com/monitisexchange/Python-SDK.git
cd Python-SDK
sudo python setup.py install

Monitis API Keys

Before you can run the tools, you will also need to set up the Monitis API Key and Secret Key. By default, these are read from the following environment variables, which you can set on the command line, or add to your .bashrc or other login command file. The values for these keys can be retrieved via the web user interface under Tools -> API -> API Key.

# production keys
export MONITIS_APIKEY='API KEY'
export MONITIS_SECRETKEY='SECRET KEY'
# sandbox keys
export MONITIS_SANDBOX_APIKEY='SANDBOX API KEY'
export MONITIS_SANDBOX_SECRETKEY='SANDBOX SECRET KEY'

Conclusion

While the Monitis REST API makes creating custom monitors simple in almost any language, it becomes truly easy with an SDK to handle the REST interactions. This article has illustrated just how effortless it can be to collect important information from your systems and record it in Monitis custom monitors.

Signup and use free monitis sandbox here - http://sandbox.monitis.com.

See also Python Performance Tips, Part 1

Share Now:del.icio.usDiggFacebookLinkedInBlinkListDZoneGoogle BookmarksRedditStumbleUponTwitterRSS

Read the original blog entry...

@ThingsExpo Stories
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, will provide an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profes...
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
One of the bewildering things about DevOps is integrating the massive toolchain including the dozens of new tools that seem to crop up every year. Part of DevOps is Continuous Delivery and having a complex toolchain can add additional integration and setup to your developer environment. In his session at @DevOpsSummit at 18th Cloud Expo, Miko Matsumura, Chief Marketing Officer of Gradle Inc., will discuss which tools to use in a developer stack, how to provision the toolchain to minimize onboa...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Avere delivers a more modern architectural approach to storage that doesn’t require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbuilding of data centers ...
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 ad...
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
SYS-CON Events announced today that VAI, a leading ERP software provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. VAI (Vormittag Associates, Inc.) is a leading independent mid-market ERP software developer renowned for its flexible solutions and ability to automate critical business functions for the distribution, manufacturing, specialty retail and service sectors. An IBM Premier Business Part...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, will discuss the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filte...
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
Fortunately, meaningful and tangible business cases for IoT are plentiful in a broad array of industries and vertical markets. These range from simple warranty cost reduction for capital intensive assets, to minimizing downtime for vital business tools, to creating feedback loops improving product design, to improving and enhancing enterprise customer experiences. All of these business cases, which will be briefly explored in this session, hinge on cost effectively extracting relevant data from ...
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies adopt disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advanced analytics, and DevO...
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts...
As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense. In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...
SYS-CON Events announced today that iDevices®, the preeminent brand in the connected home industry, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. iDevices, the preeminent brand in the connected home industry, has a growing line of HomeKit-enabled products available at the largest retailers worldwide. Through the “Designed with iDevices” co-development program and its custom-built IoT Cloud Infrastruc...
Eighty percent of a data scientist’s time is spent gathering and cleaning up data, and 80% of all data is unstructured and almost never analyzed. Cognitive computing, in combination with Big Data, is changing the equation by creating data reservoirs and using natural language processing to enable analysis of unstructured data sources. This is impacting every aspect of the analytics profession from how data is mined (and by whom) to how it is delivered. This is not some futuristic vision: it's ha...
Silver Spring Networks, Inc. (NYSE: SSNI) extended its Internet of Things technology platform with performance enhancements to Gen5 – its fifth generation critical infrastructure networking platform. Already delivering nearly 23 million devices on five continents as one of the leading networking providers in the market, Silver Spring announced it is doubling the maximum speed of its Gen5 network to up to 2.4 Mbps, increasing computational performance by 10x, supporting simultaneous mesh communic...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including clou...
Most people haven’t heard the word, “gamification,” even though they probably, and perhaps unwittingly, participate in it every day. Gamification is “the process of adding games or game-like elements to something (as a task) so as to encourage participation.” Further, gamification is about bringing game mechanics – rules, constructs, processes, and methods – into the real world in an effort to engage people. In his session at @ThingsExpo, Robert Endo, owner and engagement manager of Intrepid D...