Welcome!

Java IoT Authors: Elizabeth White, Stefan Bernbo, Harry Trott, Liz McMillan, Pat Romanski

Blog Feed Post

Using the Python SDK for Monitis Custom Monitors

One advantage of using Monitis to monitor your systems and applications is the flexibility to use either the native agent or custom monitor code written in virtually any language. For custom monitors, the REST API provides the basic foundation to interface programmatically with Monitis. For many popular languages, there are open source SDKs available to make the process of interfacing with Monitis even easier. You can find links to Java, Perl, PHP, Ruby, C#, PowerShell, and VisualBasic SDKs and example scripts at http://monitis.com/api/api.html#sdk.

The Monitis Sandbox

For this article, we’ll be using the Monitis sandbox, a free development environment available to aid in the creation of Monitis custom monitors. You can sign up for an account at http://sandbox.monitis.com/.

Netstat

Netstat is a command available on unix systems to provide information about the state of the network interfaces. It has several modes, one of which can display statistics about the number of packets and datagrams sent and received. While there is no common API across unix systems to access this data, it is possible to extract this information from the text output of the program. Since the output varies slightly from system to system, it is necessary to take these variations into account when using that output.

Netstat examples using the Python Monitis SDK

The examples for this article are available at https://github.com/monitisexchange/Python-SDK/tree/master/examples. There are two examples. The first, monitis_netstat.py, can be used as is to create, list, update, and delete netstat monitors. The second, monitis_netstat_minimal.py, is a minimal implementation intended to illustrate the concepts for this article, but lacking the structure to handle command line arguments, different operating systems, etc.

monitis_netstat_minimal.py

Let’s take a look at monitis_netstat_minimal.py, line by line. It illustrates many of the core concepts in the Python SDK.

The following lines are imports from the Python standard library, used for calling netstat, searching its output, and comparing output from netstat with that retrieved via the SDK.

from subprocess import Popen, PIPE
from re import findall
from datetime import datetime

Next are the imports from the Monitis Python SDK itself, used for creating new monitors and fetching existing monitors.

from monitis.monitors.custom import CustomMonitor, get_monitors
from monitis.monitors.params import ResultParams, DataType
from monitis.api import Monitis

The Monitis.sandbox flag causes the SDK to use the Monitis developer’s sandbox, http://sandbox.monitis.com. By default, when this flag isn’t set, it uses the production environment, http://www.monitis.com.

Monitis.sandbox = True

The following block creates a new monitor. ResultParams encapsulates the format for the parameters that a monitor accepts. One or more of them can be passed in to the next line. CustomMonitor.addMonitor takes the parameters and creates a monitor based on them. It returns a new CustomMonitor object that represents the monitor in the Monitis API.

rp = ResultParams(
    'tcp_packets_in','TCP Packets In','pkts/s', DataType('integer'))
cm = CustomMonitor.add_monitor(rp, name='netstat monitor', tag='netstat')
print "Created monitor: %s" % cm.monitor_id

This code extracts the interesting data from the netstat output. In this case, that data is the count of TCP packets received. This is the only block of code in the minimal sample program that really isn’t dependent on the API. A different monitor could get data in any number of ways. Note that this call to netstat is based on OS X style output, but only a small change to the command and matching string is required to adapt it to virtually any unix-like operating system.

subproc = Popen(
    'netstat -s -p tcp | grep "packets received$"', shell=True, stdout=PIPE)
(netstat_out, netstat_err) = subproc.communicate()
count = findall('(\d+) packets received', netstat_out)[0]

Next, add_result takes the new data, and posts it in the format defined earlier in ResultParams.

cm.add_result(tcp_packets_in=count)

Here, we check that retrieving the results from the monitor gets back the same data that we put in.

today = datetime.utcnow()
result = cm.get_monitor_results(today.year, today.month, today.day)

Finally, delete the monitor. This exists in this example so that it cleans up after itself and doesn’t cause unused custom monitors to pile up.

cm.delete_monitor()

In typical usage, a monitor is only deleted when that data is no longer needed. This principle is explored further in monitis_netstat.py.

monitis_netstat.py

Under normal circumstances, a monitor is created once, data is updated on a periodic basis over time, and then that data is viewed in the Monitis web interface. To ilustrate this, monitis_netstat.py has command line flags to create, list, delete, and update a monitor.

Create

Create a new netstat monitor:

$ examples/monitis_netstat.py -s -c netstat_mon
Using name netstat_mon
Created new netstat monitor

List

List the monitors to verify that ours was created:

$ examples/monitis_netstat.py -s -l
2267	netstat_mon

Update

Run the monitor 10 times over five minutes to gather some data to display.

$ while (true) do examples/monitis_netstat.py -s -u 2267; sleep 30; done
1253222, 1616400, 381497, 551524
1253393, 1616514, 381505, 551536
1253436, 1616553, 381512, 551545
1253468, 1616580, 381514, 551549
1253485, 1616596, 381517, 551558
1253526, 1616633, 381522, 551609
1253564, 1616666, 381531, 551622
1253599, 1616695, 381536, 551629
1253623, 1616716, 381538, 551633
1253647, 1616740, 381544, 551638

Once this is complete, the results can be seen in the web interface.

Delete

$ examples/monitis_netstat.py -s -d 2267
Monitor 2267 deleted
$ examples/monitis_netstat.py -s -l

Try it

To get the code and try this yourself, you can clone a copy from GitHub, install the Monitis SDK module, and run it on your own system.

git clone git://github.com/monitisexchange/Python-SDK.git
cd Python-SDK
sudo python setup.py install

Monitis API Keys

Before you can run the tools, you will also need to set up the Monitis API Key and Secret Key. By default, these are read from the following environment variables, which you can set on the command line, or add to your .bashrc or other login command file. The values for these keys can be retrieved via the web user interface under Tools -> API -> API Key.

# production keys
export MONITIS_APIKEY='API KEY'
export MONITIS_SECRETKEY='SECRET KEY'
# sandbox keys
export MONITIS_SANDBOX_APIKEY='SANDBOX API KEY'
export MONITIS_SANDBOX_SECRETKEY='SANDBOX SECRET KEY'

Conclusion

While the Monitis REST API makes creating custom monitors simple in almost any language, it becomes truly easy with an SDK to handle the REST interactions. This article has illustrated just how effortless it can be to collect important information from your systems and record it in Monitis custom monitors.

Signup and use free monitis sandbox here - http://sandbox.monitis.com.

See also Python Performance Tips, Part 1

Share Now:del.icio.usDiggFacebookLinkedInBlinkListDZoneGoogle BookmarksRedditStumbleUponTwitterRSS

Read the original blog entry...

@ThingsExpo Stories
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2016 Silicon Valley. The 19th Cloud Expo and 6th @ThingsExpo will take place on November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. "The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Interne...
"We work in the area of Big Data analytics and Big Data analytics is a very crowded space - you have Hadoop, ETL, warehousing, visualization and there's a lot of effort trying to get these tools to talk to each other," explained Mukund Deshpande, head of the Analytics practice at Accelerite, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
The idea of comparing data in motion (at the sensor level) to data at rest (in a Big Data server warehouse) with predictive analytics in the cloud is very appealing to the industrial IoT sector. The problem Big Data vendors have, however, is access to that data in motion at the sensor location. In his session at @ThingsExpo, Scott Allen, CMO of FreeWave, discussed how as IoT is increasingly adopted by industrial markets, there is going to be an increased demand for sensor data from the outermos...
UAS, drones or unmanned aircraft, no matter what you call them — this was their week. Our news stream was flooded with updates on the newly announced rules and regulations for commercial UAS from the FAA. So, naturally we have dedicated this week’s top news round up to highlight some of our favorite UAS stories.
Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2016 Silicon Valley. The 6thInternet of @ThingsExpo will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
CenturyLink has announced that application server solutions from GENBAND are now available as part of CenturyLink’s Networx contracts. The General Services Administration (GSA)’s Networx program includes the largest telecommunications contract vehicles ever awarded by the federal government. CenturyLink recently secured an extension through spring 2020 of its offerings available to federal government agencies via GSA’s Networx Universal and Enterprise contracts. GENBAND’s EXPERiUS™ Application...
In addition to all the benefits, IoT is also bringing new kind of customer experience challenges - cars that unlock themselves, thermostats turning houses into saunas and baby video monitors broadcasting over the internet. This list can only increase because while IoT services should be intuitive and simple to use, the delivery ecosystem is a myriad of potential problems as IoT explodes complexity. So finding a performance issue is like finding the proverbial needle in the haystack.
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...
Presidio has received the 2015 EMC Partner Services Quality Award from EMC Corporation for achieving outstanding service excellence and customer satisfaction as measured by the EMC Partner Services Quality (PSQ) program. Presidio was also honored as the 2015 EMC Americas Marketing Excellence Partner of the Year and 2015 Mid-Market East Partner of the Year. The EMC PSQ program is a project-specific survey program designed for partners with Service Partner designations to solicit customer feedbac...
Apixio Inc. has raised $19.3 million in Series D venture capital funding led by SSM Partners with participation from First Analysis, Bain Capital Ventures and Apixio’s largest angel investor. Apixio will dedicate the proceeds toward advancing and scaling products powered by its cognitive computing platform, further enabling insights for optimal patient care. The Series D funding comes as Apixio experiences strong momentum and increasing demand for its HCC Profiler solution, which mines unstruc...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
IoT offers a value of almost $4 trillion to the manufacturing industry through platforms that can improve margins, optimize operations & drive high performance work teams. By using IoT technologies as a foundation, manufacturing customers are integrating worker safety with manufacturing systems, driving deep collaboration and utilizing analytics to exponentially increased per-unit margins. However, as Benoit Lheureux, the VP for Research at Gartner points out, “IoT project implementers often ...
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
Connected devices and the industrial internet are growing exponentially every year with Cisco expecting 50 billion devices to be in operation by 2020. In this period of growth, location-based insights are becoming invaluable to many businesses as they adopt new connected technologies. Knowing when and where these devices connect from is critical for a number of scenarios in supply chain management, disaster management, emergency response, M2M, location marketing and more. In his session at @Th...
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to imp...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...