Welcome!

Java IoT Authors: APM Blog, Jnan Dash, Elizabeth White, Stackify Blog, Liz McMillan

Related Topics: Java IoT

Java IoT: Article

Whole House Audio from the Palm of Your Hand

Whole House Audio from the Palm of Your Hand

Parts 1 and 2 of this series (JDJ, Vol. 7, issues 6 and 9) demonstrated how I developed a remote control MP3 player by using a Bluetooth connection from my handheld (a Compaq iPaq) and employing a client and a server written in Java.

Tracks were weighted to increase (or decrease) their chances of playing, and each track had a preferred track that followed, so the system would generally play two or three tracks from a particular album before moving on to the next one. The system works pretty well, even though a recent ripping session increased the tracks available to around 3,000 ­ remotely controlling my listening is great, but not perfect.

Like many people, I have speakers dotted around my house, with two amplifiers (one upstairs, another downstairs) and switches everywhere to control where the sound actually comes out. This is fine, but remembering which switch does what is a pain. If I'm in bed I don't want to have to wander into the office to switch on the speakers in the bedroom, so it's time to add some speaker control to my setup.

In this article I'll show how a Java application can easily take control of physical systems with the right hardware, both controlling hardware and responding to real-world events (well, a doorbell), and, as this is the final installment, I'll also be talking a little about why I developed this application and where it will go next. The source code can be downloaded from www.sys-con.com/java/sourcec.cfm.

Controlling the Speakers
First I had to sort out how I was actually going to control my speakers from the computer. I hung up my soldering iron years ago, so I wasn't about to start building my own circuits ­ luckily a company called Quasar already makes the perfect circuit for my needs. They actually sell several kits that would work, but I went for a serial-controlled box with eight relays and four inputs. I also didn't fancy putting the thing together myself, so I ordered it preassembled, just plug and go. Testing via Windows HyperTerminal is easy and quick, as the commands sent to the box are simple indeed (N1 turns on relay one; F1 turns off relay one).

Originally, I wired each relay to the same 9-volt supply that provides power to the box, using that voltage to switch two dual-poll/dual-throw relays, one for each speaker. With this arrangement, one relay on the box controlled one set of speakers, but also required a lot of wires and didn't work satisfactorily. So I simplified it by wiring each speaker through a relay on the box, one relay for each speaker, using up six of my eight available relays (for three sets of speakers).

Audiophile friends have pointed out that I should really be smoothing the output to avoid the coil in my speakers pulling a high load on connection (which might cause "popping" when the speakers are connected), and Quasar does sell a kit to handle this. However, the speakers in the bedroom are nice NXT flat speakers without a coil, and the other sets are cheap, so I'll see how it goes. Pretty quickly (once I had given up my extra relays) I had speakers I could switch on and off from HyperTerminal.

The relays in the control box will actually handle the mains voltage, opening up all sorts of interesting ideas. Controlling the doors and lights has been vetoed by the house's other resident on the grounds that a crashed computer could plunge the house into darkness, but there's certainly room for a few more boxes in the future.

Then it was just a matter of writing a server application that would listen to one serial port for incoming commands over the Bluetooth connection and on receipt send them to the relay control box. I first developed the output side, and stuck a GUI on the application so I could control the speakers from the server screen.

One of the great things about object-oriented development is this ability to add and remove components easily. While the development could have been tested using command-line arguments, it was quicker and easier to stick a user interface on it with a couple of buttons that could be removed later. During testing I discovered that for some reason it would switch only one speaker in each pair, despite the fact that it was sending the right commands. It was quite a long time before I finally realized that the relay control box just couldn't respond to the commands fast enough. Putting in a Thread.sleep() command sorted the problem, but had the side effect of turning on a pair of speakers one at a time, which can sound a little odd. I guess I'll be digging out those extra relays at some point.

As mentioned in Part 2, the Bluetooth stack not only allows serial connectivity, but also maps two standard serial ports to the Bluetooth network. Using the TDK USB Bluetooth on the server means that COM4 represents an incoming Bluetooth connection, so this is what we'll be listening to. As I had already created a "SerialConnection" class to communicate with the relay control box (using COM1), it made sense to just instantiate another instance of that class to handle the Bluetooth side of the process. I then discovered to my horror that my application had stopped working completely.

The problem was eventually narrowed to the opening of the serial port, and though I had not made any changes to the code, it was now apparently finding four separate ports, all called COM1! The only thing I had done since it worked was add the Bluetooth hardware, so for some reason that was causing the problem. It turned out that I was only checking if the name of the port found started with "COM1", and the Bluetooth drivers had added COM11, COM12, and COM13, all of which were considered fair game by my application. This kind of problem is common enough when applications are moved between systems and should be guarded against; loops searching for things should stop when they've found them!

Once that was fixed, I instantiated two instances of my SerialConnection object and started testing, opening a serial connection from another PC with Bluetooth, again using HyperTerminal for testing. I decided on a very basic protocol with no perceptible handshaking at all. The client simply sent the name of the room followed by "on" or "off" and the server would switch the speakers.

Some sort of response from the server would have been better, and should probably be added for robustness, but for the moment I'm relying on any transmission problems being reported by the serial connection. Initial testing went well, except that the switching commands were being sent back to the PC making the request instead of to the relay-switching box. Again my lack of architecture design let me down, and what I had were two instances of my SerialConnection class communicating with devices with no way to communicate with each other. So I overloaded the constructor with one SerialConnection instance that would take the other as a parameter to be used to report when requests were made:

SerialConnection(CommPortIdentifier portId,
SerialConnection localCom) {
this(portId);
outputConnection = localCom;
isBluetoothLink = true;
}

The Boolean "isBluetoothLink" is then used as a marker, so only the Bluetooth connection responds to messages received (see Listing 1). The strange arrangement whereby the office is on switches one and seven is due to physical difficulties with the lengths of wire available.

That got the server working, so I could start on the client. I was very keen on having a graphical interface, so knocked up a floor plan of the upstairs of my house (see Figure 1) so I could tap on a room to switch the speakers; it was a matter of putting the graphic on the screen and having it respond to taps in the right place.

Of course, PersonalJava doesn't support serial connections so I used James Nord's excellent implementation, which provides all the important bits of the Java Communications API for the Pocket PC. The outgoing serial port on the iPaq is COM8, and, as previously discussed, if COM8 is opened without a connection, the Bluetooth stack will reestablish a connection to the last place it was connected to. So I manually connected to my server one time, and can now run the application fine (until I use the serial port to connect somewhere else).

Testing the Application
Testing was remarkably smooth, with only one issue ­ the application could only be run once and the iPaq had to be reset before it could be used again. Obviously something was not being closed properly, and the serial ports were prime suspects; sure enough, explicitly closing the port sorted out that problem.

It would be nice if the application polled the relay box to get the settings so the display could reflect the sound coming out when it was launched, but I can normally hear if the speakers are on, so it's not a priority. The display isn't double buffered so there is some flicker, but that's not important in an application like this. The connection is fast though, taking a few seconds, and it's nice to be able to wander around the house, switching the music to follow.

The speaker control program is called "Conductor", but is still a separate application from the MP3 player. Since I want them both to respond to Bluetooth connections and run on the same machine, I combined them into a single application. This turned out to be remarkably simple, both on the server and the client sides. The MP3 player class simply instantiates an instance of Conductor and passes it any unrecognized commands received over the network, while Conductor does the same with commands received over Bluetooth, passing them on to the MP3 player. This also meant that Conductor would work over a network connection, making it much easier to combine the client applications.

The parent class of the Conductor client was just changed from a frame to a dialog (as only one frame can be used per application in J2ME) and instantiated from the MP3 client. I also took the opportunity to move some buttons around on the MP3 client that were vanishing under the Jeode control bar on the iPaq. Now I had one client and one server talking over Bluetooth or network connections (specified on the client command line).

The Doorbell Saga
We moved into our current house about a year ago and have never had a doorbell. We always felt that it should be something silly, but never saw anything that suited us, so this seemed the ideal time to look at the inputs on the relay control box. Since they can detect incoming voltage, I thought it might be interesting to connect a doorbell that could then control the sound playback. A cheap wireless doorbell turned out to be only a tenner and was remarkably easy to wire in; I just disconnected the speaker and ran the wires into the sensor on the relay control box. By polling the box with "I1", it will return "0" if nothing is happening and "1" if the doorbell is trying to chime. Since the chime lasts a couple of seconds there's no hurry in polling.

One architectural issue that came back to hurt me was the fact that I had always used event-driven serial communications. What this means is that the client makes a request of the server, but doesn't wait for a response. When the server sends its response, the client reacts to it without knowing that it was responsible for sending the request. This makes for quick code development, and if the responses from the server can be easily identified, then it's a fair way of working.

Now I was communicating with the relay control box, which has only very basic responses and it's much harder to identify what it's responding to. For that reason I needed to block until a response was received from the control box, so I had to change the event mechanism. It just stores the incoming message in a string and creates a method that checks the string and blocks until a message is received (see Listing 2).

The getMessage method also ignores messages starting with "#", as these are echoes of what was sent.

Polling the input in another thread is very easy, and when a "1" is discovered, the music is stopped, all the speakers in the house are switched on, and a predetermined MP3 file is played; then the speakers are switched back to their previous state and the music restarts if it was paused earlier. Listing 3 shows the polling loop.

Note that only one thread can talk to the relay control box at a time, so a Boolean is set if the speakers are being turned on or off (remember that due to the speed of the relay control box this will take half a second) and a poll might be missed. This works pretty well, though I did have to introduce another Boolean to stop the speakers from trying to be controlled while this thread is polling, and my head started to hurt.

The doorbell will play the file "door.mp3" when the doorbell is pressed, and for the moment this is the sound of an old-fashioned knocker banging against wood, but there is still an issue with range. Wireless technology is not nearly as common in the UK as in the States; basically, due to the limited amount of space available, the frequencies we can use are much more limited and transmissions are lower power.

The wireless doorbell I had bought claimed to have a range of 30 meters, which should easily encompass my house (Bluetooth works just about house-wide, and it's only supposed to be 10 meters) but I was horrified to discover that my 10 pounds worth of wireless doorbell actually has a range of about 2 meters. To ring the doorbell you have to invite visitors in and show them halfway up the stairs, where they can press the bell and be impressed. While this amused my wife a great deal, it's not really what I was after, so another doorbell was obtained, this time for 40 quid, which claimed to have a 100-meter range. I don't know where these figures come from, but this new doorbell actually has a range of about 4 meters, at best (admittedly, we are in the middle of a very humid heat wave at the moment, but from 100 meters down to 4 is still very poor). So some wiring was required, and now the doorbell is half wireless (as far as the bedroom) and half wired (into the office) and works properly. Relying on a server running Windows to provide a doorbell does worry me, but if we don't respond people will probably just bang on the door anyway so it's still not mission-critical code.

Of course, this left me with another wireless doorbell with an, almost, useless range, but easy to integrate with the rest of the system. Thinking it through, the place where control was most needed was the bathroom, where you might be lounging in the bath and decide some musical accompaniment would be good. Taking an iPaq into the bath is something I have done, but it will end in tears, so a cheaper alternative would be good. So the original cheap doorbell was wired into the second input on the control box, and the code expanded to poll it too and toggle the bathroom speakers when it is pressed. I'm rather taken with this idea, and may well buy another switch or two to control other functions.

Conclusion
Now I can play music to suit my mood in the rooms of my choice, though the whole application is not yet perfect. The MP3 player leaks memory somewhere, causing it to crash after about 30 hours, and it still has an obsession with "Mr. President" by 4 Non Blonds for reasons I've yet to establish, but it is a program I use almost every day and I expect it to be available whenever I want it.

The process of developing this application has also been interesting, personally, as a reminder of why I got interested in computer programming in the first place. While many people now decide to study software development as a career choice, I got interested in programming because computers weren't doing the things I wanted them to.

In these days of supercomplex software and massively available shareware, it's easy to find a program that will do pretty much what you need without going through the effort of writing it. I admit that before this project I hadn't written any software for myself for years, just accepting the limitations of what was available. But writing software for fun is actually better than I remembered. If you don't have to worry about deadlines or insane functional requirements from inept management, or don't have to document everything, it's possible to concentrate on getting things the way you want and to deal with the intellectual challenge of solving the problems, which is fun.

Next I want to spread my control to the downstairs, but laying even more cable is something I want to avoid (the house is already looking like a slab of Swiss cheese). Some sort of Bluetooth link to another machine downstairs might be the solution, perhaps a Palm Pilot, which could then control the speakers downstairs. It also occurs to me that there's a WAP profile in the Bluetooth specification that would provide the perfect interface for this kind of project and should then work with any WAP device. It would certainly be an interesting project. Fixing memory leaks and dealing with minor bugs is something I'll save for the software I'm paid to write ­ unless, of course, I get really bored.

Useful URLs

  • www.quasarelectronics.com
  • www.bluetooth.com
  • More Stories By Bill Ray

    Bill Ray, former editor-in-chief (and continuing distinguished contributor to) Wireless Business & Technology magazine, has been developing wireless applications for over 20 ears on just about every platform available. Heavily involved in Java since its release, he developed some of the first cryptography applications for Java and was a founder of JCP Computer Services, a company later sold to Sun Microsystems. At Swisscom he was responsible for the first Java-capable DTV set-top box, and currently holds the position of head of Enabling Software at 02, a UK network operator.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
    "MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    "Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
    "IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
    "Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
    Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
    Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
    "Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
    "There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
    SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
    It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
    WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
    A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
    SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
    Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
    To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
    An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics gr...