Welcome!

Java IoT Authors: Liz McMillan, Elizabeth White, Pat Romanski, Stackify Blog, Progress Blog

Related Topics: Java IoT

Java IoT: Article

Whole House Audio from the Palm of Your Hand - Part 2 of 3

Whole House Audio from the Palm of Your Hand - Part 2 of 3

In Part 1 of this series (JDJ, Vol. 7, issue 6), I showed how I developed an MP3 player in Java, and then added the ability to control that player from a wireless handheld device using a PersonalJava application.

While I could only stop, pause, adjust the volume, and select the next track to be played, I still found the application useful, but not yet perfect...

The first problem to be addressed was the combinations you can get when listening to your entire music collection at random. When a nice relaxing Enya track fades out and you find yourself launching into the world of Eminem, the shock can be considerable, not to mention that sometimes I'm just not in the mood for Aztec Camera...but have no preference beyond that. So, some sort of weighting is needed.

My next problem was the networking side of things. Wireless Ethernet is nice, but not ideal on the iPaq. In addition to the devastating impact it has on the battery life, I can't get it to connect automatically on demand. This is because I use a GPRS phone for my mobile Internet access (thus the method of connectivity must be specified) and the networking interface on Pocket PC devices isn't very good. So when I'm using the application, my iPaq has to be on all the time with the drain of networking, or I have to manually connect to the network whenever I want to control the music. Neither option is useful, and as my iPaq has Bluetooth built-in (it's a 3870), this should provide an ideal wireless mechanism.

Listening Preferences
Starting with the weighting, I decided that each track needed a descriptive profile. Since I don't want to type these in when I rip the tracks, there need to be default values that can be adjusted later. I want to be able to adjust the settings of the current track from the handheld, so if I'm listening to a track and decide I like it a lot, I can increase the chances of it playing again, or if I don't like it, downgrade it so it's unlikely to pop up later.

After some consideration I came up with a few criteria. Each track would have the following fields, rated from 1 to 100: how much I like the track, how fast it is, how loud, how instrumental, and the name of the track that should follow. The last field was put in as I have some tracks that should really be put together to make sense. The fast and loud fields refer to how energizing and rousing I feel the music is - I like loud music in the morning, fast music when I'm sewing, and instrumental music when I'm programming. With this in mind the user must be able to adjust his or her current listening preferences, based on the same fields (though not "liked," obviously).

Therefore, the process of deciding on the next track will be as follows: once a track starts playing, the track specified in the "preferred next track" field will be located and passed the current listening preferences to see if it matches them. If it does, it will be played next. If it doesn't match well enough (with a random element), another track will be selected at random and asked if it matches the preferences. By default the preferred next track is the following track on the album, though the user can change this along with the other preferences. This way several tracks can be selected until one that matches what the user wants to hear is found.

Note that a random element is also introduced, so even tracks that are not liked much can pop up on occasion. Of course, if I really hated them, I could always delete the files. I also decided that it made sense to reduce the "liked" field by one point each time the track was played, and increase it by a point each time the track was rejected. This way my collection should, in theory, rotate gently and ensure I get to hear everything over time. After some discussion, I also included a Boolean field called "aletiaApproved". This notes if the track is liked by my wife and allows me to specify that only such tracks should be played. My music tastes are fairly broad and not shared by everyone, so this also provides a general liked-by-most-people criterion.

Originally, I thought I would just serialize the Track objects into a file and read them back, but this was extremely optimistic. The Track objects contain media players and all sorts of objects that can't be serialized. I ended up creating a new class just to contain the track details; this can be constructed from a Track object or by specifying the field values as parameters. This means that when I wish to save the values (when the application is shut down), I need to create an ArrayList of TrackDetails objects with an entry for each track, and then serialize this to the local disk.

This worked fine, but loading was more complex as I not only had to load the file and create the Track objects from it, I also had to add any new tracks that had been created and remove any that had been deleted. I decided to store the details in the same directory that was selected for playback; this way I could have multiple files containing track details without worrying about which one was being used.

The application therefore looks in the current directory to see if such a file exists. If so, it loads it, then goes through its normal scanning process. Before adding any MP3 files to the main playlist, it checks to see if it already has details for that track (by comparing the full file name and path) - if it does, it uses the loaded details, if not, it creates a default set. The track is then added to the main ArrayList. This enabled me to automatically add new tracks as well as remove tracks that had been deleted since the last time the application was run, since the loaded details are only copied to the live playlist when the file is located - tracks for which there are no files are automatically removed.

I also added a directory to my MP3 collection called NoRandomPlay. MP3 files in this directory would never be played at random, but could still be selected. I did this so my Psion Digital Radio can save recordings of documentaries and plays, which are not really suitable for randomly mixing with my music but should still be selectable on demand. Ideally, I'd like to be able to control my digital radio from my mobile, but that will have to wait for another series of articles.

Network Protocol
Having decided how the application was going to work, it was time to decide how to extend the network protocol to add this functionality. Again, it's always best to start with the server so it can easily be tested using HyperTerminal or something similar. It's clear that I'll need four commands in total, two for getting and setting the current listening preferences and two for getting and setting the preferences for a particular track. Given the noncritical nature of the application, I'm not too worried about the bandwidth used, and I'm making the whole thing case insensitive, as I did before:

[client] Get Preferences
[server] +Preferences Follow
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approval: true
[client] Set Preferences:50#50*50-true
[server] +Preferences set

These commands will be used for getting and setting the current listening preferences. The latter of these deserves some explanation: the setting is done in a single line, with the preferences being specified as loud, fast, instrumental, and "Aletia approved", with different dividers marking the space between them. Using different markers makes it harder for a human to read, but much easier to parse for the application; for example, to extract the loud field from an incoming preference the following line can be used:

int loud = new Integer(message.substring(message.indexOf(',')
+ 1, message.indexOf('#'))).intValue();

Basically this is laziness on my part. It would have been better to have a properly human-readable protocol both ways, but I didn't bother. Setting and getting the preferences for a particular track is almost identical:

[client] Style 3
[server] +Style Follows
[server] Number: 3
[server] Liked: 30
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approved: true
[server] Next: 4
[client] Set_Style 3:30,50#50*50-true=12
+Style set

One additional field is specified: how much the track is liked. The number of the track is also returned; this is where the track lies in the ArrayList on the server and was included in case I ever wanted to set the style based on the file name, though that hasn't happened yet.

After I got the server working (so few words, so much work!) and tested with HyperTerminal, I started thinking about how to manage the user interface. (If this had been a commercial application, I would have probably started with the interface, but as the whole thing is intended only for my use, it was left until the end.) PersonalJava can support only one frame and one dialog, but that applies only to those concurrently visible. I'm already using one dialog for my track listing, so I created another two to set listening preferences and set the preferences for a track. While I probably could have used one class for all the preference settings, I decided they were sufficiently different to warrant their own classes.

I really wanted a slider to set the various levels, but there's no Swing in PersonalJava. If I wanted one I would have to write one, which I wasn't keen on doing. Searching around among my old work I discovered a slider object I wrote years ago, before Swing existed, which suited me completely. As the class was so old, it was able to work with PersonalJava and looked considerably better than what I would have created this time around (see Figure 1). It's always nice to reuse a component, especially one you can't remember creating.

With my sliders it was fairly easy to put together the interface for setting track and listening preferences. When the user wants to set preferences, the client asks the server for either the current listening preferences or the track preferences as requested by the user, then sets the levels of the sliders to the right place and makes the dialog visible. Changes are written back to the server when the dialog is dismissed.

The track preferences dialog didn't have the space for me to allow the user to alter the preferred next track, so that will have to wait until the next version.

Overall the system works pretty well, generally playing two or three tracks from an album before moving on to the next one, which makes for a much better listening experience. Setting the preferences has an influence, but still allows for occasional surprises, though for some strange reason the application still has a thing about "Hey Mr. President" by 4 Non Blondes.

Networking
The next problem was the networking and required a more radical solution. I liked having multiple clients, often running the client on my PC when I was working and sending the instructions over the wired LAN, so that would have to stay, but something had to be done about the mobile devices.

Bluetooth provides the ideal mechanism for such an application, with low power consumption and the ability to connect very quickly. While the iPaq is Bluetooth-capable, it needs another Bluetooth device to speak to, so I invested in a Bluetooth Network Access Point from Inventel (I did actually need it for another project too; I'm not quite that frivolous). Given the relative youth of the protocol and my urgent need (for my other project), this arrived without a manual of any sort and with drivers for another device entirely. But with the helpful support of the Inventel people I managed to get it working. It turned out to be running Linux, which proved immensely helpful in getting it working.

With a device like this I could attach my iPaq to the network and route TCP/IP packets over my LAN, or even the Internet (though I still don't have DNS working properly) via my ADSL line (see Figure 2). I have to admit I was surprised when my application ran without modification, though I shouldn't have been. While it helped, the battery consumption was still too high and I had to connect manually each time.

It's also wrong. While connecting to a LAN is useful, it's not what I wanted to do. My application wants to connect to a server; the LAN is used only as an intermediate step and is redundant. Bluetooth is about connecting devices to each other, not to networks (Wireless Ethernet is perfectly good for these kind of applications), so I should try to connect directly to the server. In that case, the server will also need Bluetooth connectivity, which was provided by a TDK Bluetooth USB Adapter (with the added benefit of allowing both my iPaq and Palm handhelds to synchronize wirelessly). Now that I had the hardware in place, it was time to think about how to get the software working with it.

One of the problems with Java, in fact, the primary problem, is its inability to support hardware it hasn't heard of. While there's a JSR for Bluetooth, it's still a work-in-progress, and it will be a while before we see any implementation, so we're on our own for the moment. However, Bluetooth was designed with compatibility in mind, and within the protocol are various "profiles" that define how different applications might use Bluetooth without getting involved with the whole radio thing. One of these profiles defines a standard for serial communications, so it should be possible to create a serial connection from the client to the server from Java. The iPaq defines two serial ports as working with Bluetooth: COM8 is used for outgoing connections while COM7 is used for incoming. On the PC these change to COM3 and COM4, respectively, though configurable through the TDK software. Of course, when you open a serial connection you don't normally specify a destination, so some experimentation was necessary to establish where an outgoing connection would end up. This showed an outgoing serial connection will connect to the last place a serial connection was made to, so getting it working was simply a matter of making the connection manually once, then running the software when required.

While this sounds good, there's one problem: serial connections are not part of PersonalJava. Not all PersonalJava devices are considered to have a serial port so we're reduced to the lowest common denominator. Luckily James Nord fixed this particular failing with a serial API that works on an iPaq and other Pocket PC devices, and is available free from his Web site (see Useful Links at the end of this article). He's never actually used it with Bluetooth, but helpfully pointed out that I was using an old version that didn't support the event-driven reception of data; it works perfectly with the latest version.

Testing the connection was relatively easy once I got the right version of the Serial API on my iPaq and the server; I used the same code on both. Opening the port is a matter of working through the available ports until one matches the desired port:

String port = "COM3";
portList = CommPortIdentifier.getPortIdentifiers();
while (portList.hasMoreElements()) {
portId = (CommPortIdentifier) portList.nextElement();
if (portId.getPortType() == CommPortIdentifier.PORT_SERIAL) {
if (portId.getName().startsWith(port)) {
CreateConnection();
}
}
}

Once the port has been identified, it's opened in the normal way:

serialPort = (SerialPort) portId.open("Test3", 2000);
inputStream = serialPort.getInputStream();
outputStream = serialPort.getOutputStream();
serialPort.addEventListener(this);
serialPort.notifyOnDataAvailable(true);

Setting the parameters is optional, since it won't matter if they don't match. Any parameters set will be used only in the connection between the Java application and the Bluetooth stack, but they can be set for completeness:

serialPort.setSerialPortParams(57600, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);

From this point it's just a matter of writing data as required and reading it back through events:

outputStream.write("My Message".getBytes());
outputStream.flush();

Remember to flush the buffer. The availability of data is indicated through a normal event, so the data can be read in (see Listing 1).

Now the client and server can communicate over the serial link, which is established within a second or two of the application being launched. The same protocols can be used, and the server simply needs to launch a second thread for listening and reporting serial communications (actually, it doesn't need to be a thread as the serial reading is not a blocking action). As the server was designed to cope with multiple clients, it has no problems with this and the application works.

Conclusion
Now I'm getting there: my application plays the music I like, tailored to fit my mood, and I can control it by simply turning on my iPaq and running the application; but the project isn't over yet. Some time ago I wired the whole house with speakers, as I like music wherever I go, but banks of switches are not ideal; more than once I've wandered upstairs to find that what we're watching on TV is being blasted in the bedroom. Now that I can control my music from my mobile, I need to start controlling the sound. In Part 3 I'll be adding the ability to turn speakers on and off from my application.

Useful Links

  • Pocket PC Serial Library: www.teilo.net/software
  • Desktop Serial Library: http://java.sun.com/products/javacomm/index.html
  • Bluetooth Bits: www.expansys.com
  • Digital Radio: www.psion.com
  • More Stories By Bill Ray

    Bill Ray, former editor-in-chief (and continuing distinguished contributor to) Wireless Business & Technology magazine, has been developing wireless applications for over 20 ears on just about every platform available. Heavily involved in Java since its release, he developed some of the first cryptography applications for Java and was a founder of JCP Computer Services, a company later sold to Sun Microsystems. At Swisscom he was responsible for the first Java-capable DTV set-top box, and currently holds the position of head of Enabling Software at 02, a UK network operator.

    Comments (1) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Ed 10/25/02 03:07:00 PM EDT

    This type of article reflects the reasons why I enjoy programming and computers in general. Its a refreshing break from the dogma wars. Thanks.

    @ThingsExpo Stories
    While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, will discuss how data centers of the future will be managed, how th...
    SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
    SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
    SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
    SYS-CON Events announced today that Keisoku Research Consultant Co. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Keisoku Research Consultant, Co. offers research and consulting in a wide range of civil engineering-related fields from information construction to preservation of cultural properties. For more information, vi...
    SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.i...
    SYS-CON Events announced today that Fusic will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Fusic Co. provides mocks as virtual IoT devices. You can customize mocks, and get any amount of data at any time in your test. For more information, visit https://fusic.co.jp/english/.
    SYS-CON Events announced today that TMC has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo and Big Data at Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Global buyers rely on TMC’s content-driven marketplaces to make purchase decisions and navigate markets. Learn how we can help you reach your marketing goals.
    SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
    There is huge complexity in implementing a successful digital business that requires efficient on-premise and cloud back-end infrastructure, IT and Internet of Things (IoT) data, analytics, Machine Learning, Artificial Intelligence (AI) and Digital Applications. In the data center alone, there are physical and virtual infrastructures, multiple operating systems, multiple applications and new and emerging business and technological paradigms such as cloud computing and XaaS. And then there are pe...
    SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak...
    Real IoT production deployments running at scale are collecting sensor data from hundreds / thousands / millions of devices. The goal is to take business-critical actions on the real-time data and find insights from stored datasets. In his session at @ThingsExpo, John Walicki, Watson IoT Developer Advocate at IBM Cloud, will provide a fast-paced developer journey that follows the IoT sensor data from generation, to edge gateway, to edge analytics, to encryption, to the IBM Bluemix cloud, to Wa...
    In his session at @ThingsExpo, Greg Gorman is the Director, IoT Developer Ecosystem, Watson IoT, will provide a short tutorial on Node-RED, a Node.js-based programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using a wide range of nodes in the palette that can be deployed to its runtime in a single-click. There is a large library of contributed nodes that help so...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
    SYS-CON Events announced today that App2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. App2Cloud is an online Platform, specializing in migrating legacy applications to any Cloud Providers (AWS, Azure, Google Cloud).
    WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, will introduce two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a mu...
    Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
    Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
    SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.