Welcome!

Java IoT Authors: Elizabeth White, Yeshim Deniz, Tim Hinds, Douglas Lyon, Stackify Blog

Related Topics: Java IoT

Java IoT: Article

Whole House Audio from the Palm of Your Hand

Whole House Audio from the Palm of Your Hand

Parts 1 and 2 of this series (JDJ, Vol. 7, issues 6 and 9) demonstrated how I developed a remote control MP3 player by using a Bluetooth connection from my handheld (a Compaq iPaq) and employing a client and a server written in Java.

Tracks were weighted to increase (or decrease) their chances of playing, and each track had a preferred track that followed, so the system would generally play two or three tracks from a particular album before moving on to the next one. The system works pretty well, even though a recent ripping session increased the tracks available to around 3,000 ­ remotely controlling my listening is great, but not perfect.

Like many people, I have speakers dotted around my house, with two amplifiers (one upstairs, another downstairs) and switches everywhere to control where the sound actually comes out. This is fine, but remembering which switch does what is a pain. If I'm in bed I don't want to have to wander into the office to switch on the speakers in the bedroom, so it's time to add some speaker control to my setup.

In this article I'll show how a Java application can easily take control of physical systems with the right hardware, both controlling hardware and responding to real-world events (well, a doorbell), and, as this is the final installment, I'll also be talking a little about why I developed this application and where it will go next. The source code can be downloaded from www.sys-con.com/java/sourcec.cfm.

Controlling the Speakers
First I had to sort out how I was actually going to control my speakers from the computer. I hung up my soldering iron years ago, so I wasn't about to start building my own circuits ­ luckily a company called Quasar already makes the perfect circuit for my needs. They actually sell several kits that would work, but I went for a serial-controlled box with eight relays and four inputs. I also didn't fancy putting the thing together myself, so I ordered it preassembled, just plug and go. Testing via Windows HyperTerminal is easy and quick, as the commands sent to the box are simple indeed (N1 turns on relay one; F1 turns off relay one).

Originally, I wired each relay to the same 9-volt supply that provides power to the box, using that voltage to switch two dual-poll/dual-throw relays, one for each speaker. With this arrangement, one relay on the box controlled one set of speakers, but also required a lot of wires and didn't work satisfactorily. So I simplified it by wiring each speaker through a relay on the box, one relay for each speaker, using up six of my eight available relays (for three sets of speakers).

Audiophile friends have pointed out that I should really be smoothing the output to avoid the coil in my speakers pulling a high load on connection (which might cause "popping" when the speakers are connected), and Quasar does sell a kit to handle this. However, the speakers in the bedroom are nice NXT flat speakers without a coil, and the other sets are cheap, so I'll see how it goes. Pretty quickly (once I had given up my extra relays) I had speakers I could switch on and off from HyperTerminal.

The relays in the control box will actually handle the mains voltage, opening up all sorts of interesting ideas. Controlling the doors and lights has been vetoed by the house's other resident on the grounds that a crashed computer could plunge the house into darkness, but there's certainly room for a few more boxes in the future.

Then it was just a matter of writing a server application that would listen to one serial port for incoming commands over the Bluetooth connection and on receipt send them to the relay control box. I first developed the output side, and stuck a GUI on the application so I could control the speakers from the server screen.

One of the great things about object-oriented development is this ability to add and remove components easily. While the development could have been tested using command-line arguments, it was quicker and easier to stick a user interface on it with a couple of buttons that could be removed later. During testing I discovered that for some reason it would switch only one speaker in each pair, despite the fact that it was sending the right commands. It was quite a long time before I finally realized that the relay control box just couldn't respond to the commands fast enough. Putting in a Thread.sleep() command sorted the problem, but had the side effect of turning on a pair of speakers one at a time, which can sound a little odd. I guess I'll be digging out those extra relays at some point.

As mentioned in Part 2, the Bluetooth stack not only allows serial connectivity, but also maps two standard serial ports to the Bluetooth network. Using the TDK USB Bluetooth on the server means that COM4 represents an incoming Bluetooth connection, so this is what we'll be listening to. As I had already created a "SerialConnection" class to communicate with the relay control box (using COM1), it made sense to just instantiate another instance of that class to handle the Bluetooth side of the process. I then discovered to my horror that my application had stopped working completely.

The problem was eventually narrowed to the opening of the serial port, and though I had not made any changes to the code, it was now apparently finding four separate ports, all called COM1! The only thing I had done since it worked was add the Bluetooth hardware, so for some reason that was causing the problem. It turned out that I was only checking if the name of the port found started with "COM1", and the Bluetooth drivers had added COM11, COM12, and COM13, all of which were considered fair game by my application. This kind of problem is common enough when applications are moved between systems and should be guarded against; loops searching for things should stop when they've found them!

Once that was fixed, I instantiated two instances of my SerialConnection object and started testing, opening a serial connection from another PC with Bluetooth, again using HyperTerminal for testing. I decided on a very basic protocol with no perceptible handshaking at all. The client simply sent the name of the room followed by "on" or "off" and the server would switch the speakers.

Some sort of response from the server would have been better, and should probably be added for robustness, but for the moment I'm relying on any transmission problems being reported by the serial connection. Initial testing went well, except that the switching commands were being sent back to the PC making the request instead of to the relay-switching box. Again my lack of architecture design let me down, and what I had were two instances of my SerialConnection class communicating with devices with no way to communicate with each other. So I overloaded the constructor with one SerialConnection instance that would take the other as a parameter to be used to report when requests were made:

SerialConnection(CommPortIdentifier portId,
SerialConnection localCom) {
this(portId);
outputConnection = localCom;
isBluetoothLink = true;
}

The Boolean "isBluetoothLink" is then used as a marker, so only the Bluetooth connection responds to messages received (see Listing 1). The strange arrangement whereby the office is on switches one and seven is due to physical difficulties with the lengths of wire available.

That got the server working, so I could start on the client. I was very keen on having a graphical interface, so knocked up a floor plan of the upstairs of my house (see Figure 1) so I could tap on a room to switch the speakers; it was a matter of putting the graphic on the screen and having it respond to taps in the right place.

Of course, PersonalJava doesn't support serial connections so I used James Nord's excellent implementation, which provides all the important bits of the Java Communications API for the Pocket PC. The outgoing serial port on the iPaq is COM8, and, as previously discussed, if COM8 is opened without a connection, the Bluetooth stack will reestablish a connection to the last place it was connected to. So I manually connected to my server one time, and can now run the application fine (until I use the serial port to connect somewhere else).

Testing the Application
Testing was remarkably smooth, with only one issue ­ the application could only be run once and the iPaq had to be reset before it could be used again. Obviously something was not being closed properly, and the serial ports were prime suspects; sure enough, explicitly closing the port sorted out that problem.

It would be nice if the application polled the relay box to get the settings so the display could reflect the sound coming out when it was launched, but I can normally hear if the speakers are on, so it's not a priority. The display isn't double buffered so there is some flicker, but that's not important in an application like this. The connection is fast though, taking a few seconds, and it's nice to be able to wander around the house, switching the music to follow.

The speaker control program is called "Conductor", but is still a separate application from the MP3 player. Since I want them both to respond to Bluetooth connections and run on the same machine, I combined them into a single application. This turned out to be remarkably simple, both on the server and the client sides. The MP3 player class simply instantiates an instance of Conductor and passes it any unrecognized commands received over the network, while Conductor does the same with commands received over Bluetooth, passing them on to the MP3 player. This also meant that Conductor would work over a network connection, making it much easier to combine the client applications.

The parent class of the Conductor client was just changed from a frame to a dialog (as only one frame can be used per application in J2ME) and instantiated from the MP3 client. I also took the opportunity to move some buttons around on the MP3 client that were vanishing under the Jeode control bar on the iPaq. Now I had one client and one server talking over Bluetooth or network connections (specified on the client command line).

The Doorbell Saga
We moved into our current house about a year ago and have never had a doorbell. We always felt that it should be something silly, but never saw anything that suited us, so this seemed the ideal time to look at the inputs on the relay control box. Since they can detect incoming voltage, I thought it might be interesting to connect a doorbell that could then control the sound playback. A cheap wireless doorbell turned out to be only a tenner and was remarkably easy to wire in; I just disconnected the speaker and ran the wires into the sensor on the relay control box. By polling the box with "I1", it will return "0" if nothing is happening and "1" if the doorbell is trying to chime. Since the chime lasts a couple of seconds there's no hurry in polling.

One architectural issue that came back to hurt me was the fact that I had always used event-driven serial communications. What this means is that the client makes a request of the server, but doesn't wait for a response. When the server sends its response, the client reacts to it without knowing that it was responsible for sending the request. This makes for quick code development, and if the responses from the server can be easily identified, then it's a fair way of working.

Now I was communicating with the relay control box, which has only very basic responses and it's much harder to identify what it's responding to. For that reason I needed to block until a response was received from the control box, so I had to change the event mechanism. It just stores the incoming message in a string and creates a method that checks the string and blocks until a message is received (see Listing 2).

The getMessage method also ignores messages starting with "#", as these are echoes of what was sent.

Polling the input in another thread is very easy, and when a "1" is discovered, the music is stopped, all the speakers in the house are switched on, and a predetermined MP3 file is played; then the speakers are switched back to their previous state and the music restarts if it was paused earlier. Listing 3 shows the polling loop.

Note that only one thread can talk to the relay control box at a time, so a Boolean is set if the speakers are being turned on or off (remember that due to the speed of the relay control box this will take half a second) and a poll might be missed. This works pretty well, though I did have to introduce another Boolean to stop the speakers from trying to be controlled while this thread is polling, and my head started to hurt.

The doorbell will play the file "door.mp3" when the doorbell is pressed, and for the moment this is the sound of an old-fashioned knocker banging against wood, but there is still an issue with range. Wireless technology is not nearly as common in the UK as in the States; basically, due to the limited amount of space available, the frequencies we can use are much more limited and transmissions are lower power.

The wireless doorbell I had bought claimed to have a range of 30 meters, which should easily encompass my house (Bluetooth works just about house-wide, and it's only supposed to be 10 meters) but I was horrified to discover that my 10 pounds worth of wireless doorbell actually has a range of about 2 meters. To ring the doorbell you have to invite visitors in and show them halfway up the stairs, where they can press the bell and be impressed. While this amused my wife a great deal, it's not really what I was after, so another doorbell was obtained, this time for 40 quid, which claimed to have a 100-meter range. I don't know where these figures come from, but this new doorbell actually has a range of about 4 meters, at best (admittedly, we are in the middle of a very humid heat wave at the moment, but from 100 meters down to 4 is still very poor). So some wiring was required, and now the doorbell is half wireless (as far as the bedroom) and half wired (into the office) and works properly. Relying on a server running Windows to provide a doorbell does worry me, but if we don't respond people will probably just bang on the door anyway so it's still not mission-critical code.

Of course, this left me with another wireless doorbell with an, almost, useless range, but easy to integrate with the rest of the system. Thinking it through, the place where control was most needed was the bathroom, where you might be lounging in the bath and decide some musical accompaniment would be good. Taking an iPaq into the bath is something I have done, but it will end in tears, so a cheaper alternative would be good. So the original cheap doorbell was wired into the second input on the control box, and the code expanded to poll it too and toggle the bathroom speakers when it is pressed. I'm rather taken with this idea, and may well buy another switch or two to control other functions.

Conclusion
Now I can play music to suit my mood in the rooms of my choice, though the whole application is not yet perfect. The MP3 player leaks memory somewhere, causing it to crash after about 30 hours, and it still has an obsession with "Mr. President" by 4 Non Blonds for reasons I've yet to establish, but it is a program I use almost every day and I expect it to be available whenever I want it.

The process of developing this application has also been interesting, personally, as a reminder of why I got interested in computer programming in the first place. While many people now decide to study software development as a career choice, I got interested in programming because computers weren't doing the things I wanted them to.

In these days of supercomplex software and massively available shareware, it's easy to find a program that will do pretty much what you need without going through the effort of writing it. I admit that before this project I hadn't written any software for myself for years, just accepting the limitations of what was available. But writing software for fun is actually better than I remembered. If you don't have to worry about deadlines or insane functional requirements from inept management, or don't have to document everything, it's possible to concentrate on getting things the way you want and to deal with the intellectual challenge of solving the problems, which is fun.

Next I want to spread my control to the downstairs, but laying even more cable is something I want to avoid (the house is already looking like a slab of Swiss cheese). Some sort of Bluetooth link to another machine downstairs might be the solution, perhaps a Palm Pilot, which could then control the speakers downstairs. It also occurs to me that there's a WAP profile in the Bluetooth specification that would provide the perfect interface for this kind of project and should then work with any WAP device. It would certainly be an interesting project. Fixing memory leaks and dealing with minor bugs is something I'll save for the software I'm paid to write ­ unless, of course, I get really bored.

Useful URLs

  • www.quasarelectronics.com
  • www.bluetooth.com
  • More Stories By Bill Ray

    Bill Ray, former editor-in-chief (and continuing distinguished contributor to) Wireless Business & Technology magazine, has been developing wireless applications for over 20 ears on just about every platform available. Heavily involved in Java since its release, he developed some of the first cryptography applications for Java and was a founder of JCP Computer Services, a company later sold to Sun Microsystems. At Swisscom he was responsible for the first Java-capable DTV set-top box, and currently holds the position of head of Enabling Software at 02, a UK network operator.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
    Nordstrom is transforming the way that they do business and the cloud is the key to enabling speed and hyper personalized customer experiences. In his session at 21st Cloud Expo, Ken Schow, VP of Engineering at Nordstrom, discussed some of the key learnings and common pitfalls of large enterprises moving to the cloud. This includes strategies around choosing a cloud provider(s), architecture, and lessons learned. In addition, he covered some of the best practices for structured team migration an...
    Product connectivity goes hand and hand these days with increased use of personal data. New IoT devices are becoming more personalized than ever before. In his session at 22nd Cloud Expo | DXWorld Expo, Nicolas Fierro, CEO of MIMIR Blockchain Solutions, will discuss how in order to protect your data and privacy, IoT applications need to embrace Blockchain technology for a new level of product security never before seen - or needed.
    Imagine if you will, a retail floor so densely packed with sensors that they can pick up the movements of insects scurrying across a store aisle. Or a component of a piece of factory equipment so well-instrumented that its digital twin provides resolution down to the micrometer.
    In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settle...
    BnkToTheFuture.com is the largest online investment platform for investing in FinTech, Bitcoin and Blockchain companies. We believe the future of finance looks very different from the past and we aim to invest and provide trading opportunities for qualifying investors that want to build a portfolio in the sector in compliance with international financial regulations.
    No hype cycles or predictions of a gazillion things here. IoT is here. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, an Associate Partner of Analytics, IoT & Cybersecurity at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He also discussed the evaluation of communication standards and IoT messaging protocols, data...
    Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
    In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, provided a fun and simple way to introduce Machine Leaning to anyone and everyone. He solved a machine learning problem and demonstrated an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intelligence and B...
    "IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
    A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
    When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
    Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
    We are given a desktop platform with Java 8 or Java 9 installed and seek to find a way to deploy high-performance Java applications that use Java 3D and/or Jogl without having to run an installer. We are subject to the constraint that the applications be signed and deployed so that they can be run in a trusted environment (i.e., outside of the sandbox). Further, we seek to do this in a way that does not depend on bundling a JRE with our applications, as this makes downloads and installations rat...
    Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
    DX World EXPO, LLC, a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
    In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
    Digital Transformation (DX) is not a "one-size-fits all" strategy. Each organization needs to develop its own unique, long-term DX plan. It must do so by realizing that we now live in a data-driven age, and that technologies such as Cloud Computing, Big Data, the IoT, Cognitive Computing, and Blockchain are only tools. In her general session at 21st Cloud Expo, Rebecca Wanta explained how the strategy must focus on DX and include a commitment from top management to create great IT jobs, monitor ...
    "Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
    The IoT Will Grow: In what might be the most obvious prediction of the decade, the IoT will continue to expand next year, with more and more devices coming online every single day. What isn’t so obvious about this prediction: where that growth will occur. The retail, healthcare, and industrial/supply chain industries will likely see the greatest growth. Forrester Research has predicted the IoT will become “the backbone” of customer value as it continues to grow. It is no surprise that retail is ...