Welcome!

Java IoT Authors: Stackify Blog, Pat Romanski, Liz McMillan, Elizabeth White, Yeshim Deniz

Related Topics: Java IoT

Java IoT: Article

Whole House Audio from the Palm of Your Hand - Part 2 of 3

Whole House Audio from the Palm of Your Hand - Part 2 of 3

In Part 1 of this series (JDJ, Vol. 7, issue 6), I showed how I developed an MP3 player in Java, and then added the ability to control that player from a wireless handheld device using a PersonalJava application.

While I could only stop, pause, adjust the volume, and select the next track to be played, I still found the application useful, but not yet perfect...

The first problem to be addressed was the combinations you can get when listening to your entire music collection at random. When a nice relaxing Enya track fades out and you find yourself launching into the world of Eminem, the shock can be considerable, not to mention that sometimes I'm just not in the mood for Aztec Camera...but have no preference beyond that. So, some sort of weighting is needed.

My next problem was the networking side of things. Wireless Ethernet is nice, but not ideal on the iPaq. In addition to the devastating impact it has on the battery life, I can't get it to connect automatically on demand. This is because I use a GPRS phone for my mobile Internet access (thus the method of connectivity must be specified) and the networking interface on Pocket PC devices isn't very good. So when I'm using the application, my iPaq has to be on all the time with the drain of networking, or I have to manually connect to the network whenever I want to control the music. Neither option is useful, and as my iPaq has Bluetooth built-in (it's a 3870), this should provide an ideal wireless mechanism.

Listening Preferences
Starting with the weighting, I decided that each track needed a descriptive profile. Since I don't want to type these in when I rip the tracks, there need to be default values that can be adjusted later. I want to be able to adjust the settings of the current track from the handheld, so if I'm listening to a track and decide I like it a lot, I can increase the chances of it playing again, or if I don't like it, downgrade it so it's unlikely to pop up later.

After some consideration I came up with a few criteria. Each track would have the following fields, rated from 1 to 100: how much I like the track, how fast it is, how loud, how instrumental, and the name of the track that should follow. The last field was put in as I have some tracks that should really be put together to make sense. The fast and loud fields refer to how energizing and rousing I feel the music is - I like loud music in the morning, fast music when I'm sewing, and instrumental music when I'm programming. With this in mind the user must be able to adjust his or her current listening preferences, based on the same fields (though not "liked," obviously).

Therefore, the process of deciding on the next track will be as follows: once a track starts playing, the track specified in the "preferred next track" field will be located and passed the current listening preferences to see if it matches them. If it does, it will be played next. If it doesn't match well enough (with a random element), another track will be selected at random and asked if it matches the preferences. By default the preferred next track is the following track on the album, though the user can change this along with the other preferences. This way several tracks can be selected until one that matches what the user wants to hear is found.

Note that a random element is also introduced, so even tracks that are not liked much can pop up on occasion. Of course, if I really hated them, I could always delete the files. I also decided that it made sense to reduce the "liked" field by one point each time the track was played, and increase it by a point each time the track was rejected. This way my collection should, in theory, rotate gently and ensure I get to hear everything over time. After some discussion, I also included a Boolean field called "aletiaApproved". This notes if the track is liked by my wife and allows me to specify that only such tracks should be played. My music tastes are fairly broad and not shared by everyone, so this also provides a general liked-by-most-people criterion.

Originally, I thought I would just serialize the Track objects into a file and read them back, but this was extremely optimistic. The Track objects contain media players and all sorts of objects that can't be serialized. I ended up creating a new class just to contain the track details; this can be constructed from a Track object or by specifying the field values as parameters. This means that when I wish to save the values (when the application is shut down), I need to create an ArrayList of TrackDetails objects with an entry for each track, and then serialize this to the local disk.

This worked fine, but loading was more complex as I not only had to load the file and create the Track objects from it, I also had to add any new tracks that had been created and remove any that had been deleted. I decided to store the details in the same directory that was selected for playback; this way I could have multiple files containing track details without worrying about which one was being used.

The application therefore looks in the current directory to see if such a file exists. If so, it loads it, then goes through its normal scanning process. Before adding any MP3 files to the main playlist, it checks to see if it already has details for that track (by comparing the full file name and path) - if it does, it uses the loaded details, if not, it creates a default set. The track is then added to the main ArrayList. This enabled me to automatically add new tracks as well as remove tracks that had been deleted since the last time the application was run, since the loaded details are only copied to the live playlist when the file is located - tracks for which there are no files are automatically removed.

I also added a directory to my MP3 collection called NoRandomPlay. MP3 files in this directory would never be played at random, but could still be selected. I did this so my Psion Digital Radio can save recordings of documentaries and plays, which are not really suitable for randomly mixing with my music but should still be selectable on demand. Ideally, I'd like to be able to control my digital radio from my mobile, but that will have to wait for another series of articles.

Network Protocol
Having decided how the application was going to work, it was time to decide how to extend the network protocol to add this functionality. Again, it's always best to start with the server so it can easily be tested using HyperTerminal or something similar. It's clear that I'll need four commands in total, two for getting and setting the current listening preferences and two for getting and setting the preferences for a particular track. Given the noncritical nature of the application, I'm not too worried about the bandwidth used, and I'm making the whole thing case insensitive, as I did before:

[client] Get Preferences
[server] +Preferences Follow
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approval: true
[client] Set Preferences:50#50*50-true
[server] +Preferences set

These commands will be used for getting and setting the current listening preferences. The latter of these deserves some explanation: the setting is done in a single line, with the preferences being specified as loud, fast, instrumental, and "Aletia approved", with different dividers marking the space between them. Using different markers makes it harder for a human to read, but much easier to parse for the application; for example, to extract the loud field from an incoming preference the following line can be used:

int loud = new Integer(message.substring(message.indexOf(',')
+ 1, message.indexOf('#'))).intValue();

Basically this is laziness on my part. It would have been better to have a properly human-readable protocol both ways, but I didn't bother. Setting and getting the preferences for a particular track is almost identical:

[client] Style 3
[server] +Style Follows
[server] Number: 3
[server] Liked: 30
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approved: true
[server] Next: 4
[client] Set_Style 3:30,50#50*50-true=12
+Style set

One additional field is specified: how much the track is liked. The number of the track is also returned; this is where the track lies in the ArrayList on the server and was included in case I ever wanted to set the style based on the file name, though that hasn't happened yet.

After I got the server working (so few words, so much work!) and tested with HyperTerminal, I started thinking about how to manage the user interface. (If this had been a commercial application, I would have probably started with the interface, but as the whole thing is intended only for my use, it was left until the end.) PersonalJava can support only one frame and one dialog, but that applies only to those concurrently visible. I'm already using one dialog for my track listing, so I created another two to set listening preferences and set the preferences for a track. While I probably could have used one class for all the preference settings, I decided they were sufficiently different to warrant their own classes.

I really wanted a slider to set the various levels, but there's no Swing in PersonalJava. If I wanted one I would have to write one, which I wasn't keen on doing. Searching around among my old work I discovered a slider object I wrote years ago, before Swing existed, which suited me completely. As the class was so old, it was able to work with PersonalJava and looked considerably better than what I would have created this time around (see Figure 1). It's always nice to reuse a component, especially one you can't remember creating.

With my sliders it was fairly easy to put together the interface for setting track and listening preferences. When the user wants to set preferences, the client asks the server for either the current listening preferences or the track preferences as requested by the user, then sets the levels of the sliders to the right place and makes the dialog visible. Changes are written back to the server when the dialog is dismissed.

The track preferences dialog didn't have the space for me to allow the user to alter the preferred next track, so that will have to wait until the next version.

Overall the system works pretty well, generally playing two or three tracks from an album before moving on to the next one, which makes for a much better listening experience. Setting the preferences has an influence, but still allows for occasional surprises, though for some strange reason the application still has a thing about "Hey Mr. President" by 4 Non Blondes.

Networking
The next problem was the networking and required a more radical solution. I liked having multiple clients, often running the client on my PC when I was working and sending the instructions over the wired LAN, so that would have to stay, but something had to be done about the mobile devices.

Bluetooth provides the ideal mechanism for such an application, with low power consumption and the ability to connect very quickly. While the iPaq is Bluetooth-capable, it needs another Bluetooth device to speak to, so I invested in a Bluetooth Network Access Point from Inventel (I did actually need it for another project too; I'm not quite that frivolous). Given the relative youth of the protocol and my urgent need (for my other project), this arrived without a manual of any sort and with drivers for another device entirely. But with the helpful support of the Inventel people I managed to get it working. It turned out to be running Linux, which proved immensely helpful in getting it working.

With a device like this I could attach my iPaq to the network and route TCP/IP packets over my LAN, or even the Internet (though I still don't have DNS working properly) via my ADSL line (see Figure 2). I have to admit I was surprised when my application ran without modification, though I shouldn't have been. While it helped, the battery consumption was still too high and I had to connect manually each time.

It's also wrong. While connecting to a LAN is useful, it's not what I wanted to do. My application wants to connect to a server; the LAN is used only as an intermediate step and is redundant. Bluetooth is about connecting devices to each other, not to networks (Wireless Ethernet is perfectly good for these kind of applications), so I should try to connect directly to the server. In that case, the server will also need Bluetooth connectivity, which was provided by a TDK Bluetooth USB Adapter (with the added benefit of allowing both my iPaq and Palm handhelds to synchronize wirelessly). Now that I had the hardware in place, it was time to think about how to get the software working with it.

One of the problems with Java, in fact, the primary problem, is its inability to support hardware it hasn't heard of. While there's a JSR for Bluetooth, it's still a work-in-progress, and it will be a while before we see any implementation, so we're on our own for the moment. However, Bluetooth was designed with compatibility in mind, and within the protocol are various "profiles" that define how different applications might use Bluetooth without getting involved with the whole radio thing. One of these profiles defines a standard for serial communications, so it should be possible to create a serial connection from the client to the server from Java. The iPaq defines two serial ports as working with Bluetooth: COM8 is used for outgoing connections while COM7 is used for incoming. On the PC these change to COM3 and COM4, respectively, though configurable through the TDK software. Of course, when you open a serial connection you don't normally specify a destination, so some experimentation was necessary to establish where an outgoing connection would end up. This showed an outgoing serial connection will connect to the last place a serial connection was made to, so getting it working was simply a matter of making the connection manually once, then running the software when required.

While this sounds good, there's one problem: serial connections are not part of PersonalJava. Not all PersonalJava devices are considered to have a serial port so we're reduced to the lowest common denominator. Luckily James Nord fixed this particular failing with a serial API that works on an iPaq and other Pocket PC devices, and is available free from his Web site (see Useful Links at the end of this article). He's never actually used it with Bluetooth, but helpfully pointed out that I was using an old version that didn't support the event-driven reception of data; it works perfectly with the latest version.

Testing the connection was relatively easy once I got the right version of the Serial API on my iPaq and the server; I used the same code on both. Opening the port is a matter of working through the available ports until one matches the desired port:

String port = "COM3";
portList = CommPortIdentifier.getPortIdentifiers();
while (portList.hasMoreElements()) {
portId = (CommPortIdentifier) portList.nextElement();
if (portId.getPortType() == CommPortIdentifier.PORT_SERIAL) {
if (portId.getName().startsWith(port)) {
CreateConnection();
}
}
}

Once the port has been identified, it's opened in the normal way:

serialPort = (SerialPort) portId.open("Test3", 2000);
inputStream = serialPort.getInputStream();
outputStream = serialPort.getOutputStream();
serialPort.addEventListener(this);
serialPort.notifyOnDataAvailable(true);

Setting the parameters is optional, since it won't matter if they don't match. Any parameters set will be used only in the connection between the Java application and the Bluetooth stack, but they can be set for completeness:

serialPort.setSerialPortParams(57600, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);

From this point it's just a matter of writing data as required and reading it back through events:

outputStream.write("My Message".getBytes());
outputStream.flush();

Remember to flush the buffer. The availability of data is indicated through a normal event, so the data can be read in (see Listing 1).

Now the client and server can communicate over the serial link, which is established within a second or two of the application being launched. The same protocols can be used, and the server simply needs to launch a second thread for listening and reporting serial communications (actually, it doesn't need to be a thread as the serial reading is not a blocking action). As the server was designed to cope with multiple clients, it has no problems with this and the application works.

Conclusion
Now I'm getting there: my application plays the music I like, tailored to fit my mood, and I can control it by simply turning on my iPaq and running the application; but the project isn't over yet. Some time ago I wired the whole house with speakers, as I like music wherever I go, but banks of switches are not ideal; more than once I've wandered upstairs to find that what we're watching on TV is being blasted in the bedroom. Now that I can control my music from my mobile, I need to start controlling the sound. In Part 3 I'll be adding the ability to turn speakers on and off from my application.

Useful Links

  • Pocket PC Serial Library: www.teilo.net/software
  • Desktop Serial Library: http://java.sun.com/products/javacomm/index.html
  • Bluetooth Bits: www.expansys.com
  • Digital Radio: www.psion.com
  • More Stories By Bill Ray

    Bill Ray, former editor-in-chief (and continuing distinguished contributor to) Wireless Business & Technology magazine, has been developing wireless applications for over 20 ears on just about every platform available. Heavily involved in Java since its release, he developed some of the first cryptography applications for Java and was a founder of JCP Computer Services, a company later sold to Sun Microsystems. At Swisscom he was responsible for the first Java-capable DTV set-top box, and currently holds the position of head of Enabling Software at 02, a UK network operator.

    Comments (1) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Ed 10/25/02 03:07:00 PM EDT

    This type of article reflects the reasons why I enjoy programming and computers in general. Its a refreshing break from the dogma wars. Thanks.

    @ThingsExpo Stories
    SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
    When growing capacity and power in the data center, the architectural trade-offs between server scale-up vs. scale-out continue to be debated. Both approaches are valid: scale-out adds multiple, smaller servers running in a distributed computing model, while scale-up adds fewer, more powerful servers that are capable of running larger workloads. It’s worth noting that there are additional, unique advantages that scale-up architectures offer. One big advantage is large memory and compute capacity...
    In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), provided an overview of various initiatives to certify the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldwide re...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    No hype cycles or predictions of zillions of things here. IoT is big. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, Associate Partner at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He discussed the evaluation of communication standards and IoT messaging protocols, data analytics considerations, edge-to-cloud tec...
    Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
    SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
    Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
    Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists looked at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deliver...
    Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
    SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
    SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...
    We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
    SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
    SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
    SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
    In his session at Cloud Expo, Alan Winters, an entertainment executive/TV producer turned serial entrepreneur, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to ma...
    "When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    IoT solutions exploit operational data generated by Internet-connected smart “things” for the purpose of gaining operational insight and producing “better outcomes” (for example, create new business models, eliminate unscheduled maintenance, etc.). The explosive proliferation of IoT solutions will result in an exponential growth in the volume of IoT data, precipitating significant Information Governance issues: who owns the IoT data, what are the rights/duties of IoT solutions adopters towards t...
    With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...