Welcome!

Java IoT Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Carmen Gonzalez, Liz McMillan

Related Topics: Java IoT

Java IoT: Article

Whole House Audio from the Palm of Your Hand - Part 2 of 3

Whole House Audio from the Palm of Your Hand - Part 2 of 3

In Part 1 of this series (JDJ, Vol. 7, issue 6), I showed how I developed an MP3 player in Java, and then added the ability to control that player from a wireless handheld device using a PersonalJava application.

While I could only stop, pause, adjust the volume, and select the next track to be played, I still found the application useful, but not yet perfect...

The first problem to be addressed was the combinations you can get when listening to your entire music collection at random. When a nice relaxing Enya track fades out and you find yourself launching into the world of Eminem, the shock can be considerable, not to mention that sometimes I'm just not in the mood for Aztec Camera...but have no preference beyond that. So, some sort of weighting is needed.

My next problem was the networking side of things. Wireless Ethernet is nice, but not ideal on the iPaq. In addition to the devastating impact it has on the battery life, I can't get it to connect automatically on demand. This is because I use a GPRS phone for my mobile Internet access (thus the method of connectivity must be specified) and the networking interface on Pocket PC devices isn't very good. So when I'm using the application, my iPaq has to be on all the time with the drain of networking, or I have to manually connect to the network whenever I want to control the music. Neither option is useful, and as my iPaq has Bluetooth built-in (it's a 3870), this should provide an ideal wireless mechanism.

Listening Preferences
Starting with the weighting, I decided that each track needed a descriptive profile. Since I don't want to type these in when I rip the tracks, there need to be default values that can be adjusted later. I want to be able to adjust the settings of the current track from the handheld, so if I'm listening to a track and decide I like it a lot, I can increase the chances of it playing again, or if I don't like it, downgrade it so it's unlikely to pop up later.

After some consideration I came up with a few criteria. Each track would have the following fields, rated from 1 to 100: how much I like the track, how fast it is, how loud, how instrumental, and the name of the track that should follow. The last field was put in as I have some tracks that should really be put together to make sense. The fast and loud fields refer to how energizing and rousing I feel the music is - I like loud music in the morning, fast music when I'm sewing, and instrumental music when I'm programming. With this in mind the user must be able to adjust his or her current listening preferences, based on the same fields (though not "liked," obviously).

Therefore, the process of deciding on the next track will be as follows: once a track starts playing, the track specified in the "preferred next track" field will be located and passed the current listening preferences to see if it matches them. If it does, it will be played next. If it doesn't match well enough (with a random element), another track will be selected at random and asked if it matches the preferences. By default the preferred next track is the following track on the album, though the user can change this along with the other preferences. This way several tracks can be selected until one that matches what the user wants to hear is found.

Note that a random element is also introduced, so even tracks that are not liked much can pop up on occasion. Of course, if I really hated them, I could always delete the files. I also decided that it made sense to reduce the "liked" field by one point each time the track was played, and increase it by a point each time the track was rejected. This way my collection should, in theory, rotate gently and ensure I get to hear everything over time. After some discussion, I also included a Boolean field called "aletiaApproved". This notes if the track is liked by my wife and allows me to specify that only such tracks should be played. My music tastes are fairly broad and not shared by everyone, so this also provides a general liked-by-most-people criterion.

Originally, I thought I would just serialize the Track objects into a file and read them back, but this was extremely optimistic. The Track objects contain media players and all sorts of objects that can't be serialized. I ended up creating a new class just to contain the track details; this can be constructed from a Track object or by specifying the field values as parameters. This means that when I wish to save the values (when the application is shut down), I need to create an ArrayList of TrackDetails objects with an entry for each track, and then serialize this to the local disk.

This worked fine, but loading was more complex as I not only had to load the file and create the Track objects from it, I also had to add any new tracks that had been created and remove any that had been deleted. I decided to store the details in the same directory that was selected for playback; this way I could have multiple files containing track details without worrying about which one was being used.

The application therefore looks in the current directory to see if such a file exists. If so, it loads it, then goes through its normal scanning process. Before adding any MP3 files to the main playlist, it checks to see if it already has details for that track (by comparing the full file name and path) - if it does, it uses the loaded details, if not, it creates a default set. The track is then added to the main ArrayList. This enabled me to automatically add new tracks as well as remove tracks that had been deleted since the last time the application was run, since the loaded details are only copied to the live playlist when the file is located - tracks for which there are no files are automatically removed.

I also added a directory to my MP3 collection called NoRandomPlay. MP3 files in this directory would never be played at random, but could still be selected. I did this so my Psion Digital Radio can save recordings of documentaries and plays, which are not really suitable for randomly mixing with my music but should still be selectable on demand. Ideally, I'd like to be able to control my digital radio from my mobile, but that will have to wait for another series of articles.

Network Protocol
Having decided how the application was going to work, it was time to decide how to extend the network protocol to add this functionality. Again, it's always best to start with the server so it can easily be tested using HyperTerminal or something similar. It's clear that I'll need four commands in total, two for getting and setting the current listening preferences and two for getting and setting the preferences for a particular track. Given the noncritical nature of the application, I'm not too worried about the bandwidth used, and I'm making the whole thing case insensitive, as I did before:

[client] Get Preferences
[server] +Preferences Follow
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approval: true
[client] Set Preferences:50#50*50-true
[server] +Preferences set

These commands will be used for getting and setting the current listening preferences. The latter of these deserves some explanation: the setting is done in a single line, with the preferences being specified as loud, fast, instrumental, and "Aletia approved", with different dividers marking the space between them. Using different markers makes it harder for a human to read, but much easier to parse for the application; for example, to extract the loud field from an incoming preference the following line can be used:

int loud = new Integer(message.substring(message.indexOf(',')
+ 1, message.indexOf('#'))).intValue();

Basically this is laziness on my part. It would have been better to have a properly human-readable protocol both ways, but I didn't bother. Setting and getting the preferences for a particular track is almost identical:

[client] Style 3
[server] +Style Follows
[server] Number: 3
[server] Liked: 30
[server] Loud: 50
[server] Fast: 50
[server] Instrumental: 50
[server] Aletia Approved: true
[server] Next: 4
[client] Set_Style 3:30,50#50*50-true=12
+Style set

One additional field is specified: how much the track is liked. The number of the track is also returned; this is where the track lies in the ArrayList on the server and was included in case I ever wanted to set the style based on the file name, though that hasn't happened yet.

After I got the server working (so few words, so much work!) and tested with HyperTerminal, I started thinking about how to manage the user interface. (If this had been a commercial application, I would have probably started with the interface, but as the whole thing is intended only for my use, it was left until the end.) PersonalJava can support only one frame and one dialog, but that applies only to those concurrently visible. I'm already using one dialog for my track listing, so I created another two to set listening preferences and set the preferences for a track. While I probably could have used one class for all the preference settings, I decided they were sufficiently different to warrant their own classes.

I really wanted a slider to set the various levels, but there's no Swing in PersonalJava. If I wanted one I would have to write one, which I wasn't keen on doing. Searching around among my old work I discovered a slider object I wrote years ago, before Swing existed, which suited me completely. As the class was so old, it was able to work with PersonalJava and looked considerably better than what I would have created this time around (see Figure 1). It's always nice to reuse a component, especially one you can't remember creating.

With my sliders it was fairly easy to put together the interface for setting track and listening preferences. When the user wants to set preferences, the client asks the server for either the current listening preferences or the track preferences as requested by the user, then sets the levels of the sliders to the right place and makes the dialog visible. Changes are written back to the server when the dialog is dismissed.

The track preferences dialog didn't have the space for me to allow the user to alter the preferred next track, so that will have to wait until the next version.

Overall the system works pretty well, generally playing two or three tracks from an album before moving on to the next one, which makes for a much better listening experience. Setting the preferences has an influence, but still allows for occasional surprises, though for some strange reason the application still has a thing about "Hey Mr. President" by 4 Non Blondes.

Networking
The next problem was the networking and required a more radical solution. I liked having multiple clients, often running the client on my PC when I was working and sending the instructions over the wired LAN, so that would have to stay, but something had to be done about the mobile devices.

Bluetooth provides the ideal mechanism for such an application, with low power consumption and the ability to connect very quickly. While the iPaq is Bluetooth-capable, it needs another Bluetooth device to speak to, so I invested in a Bluetooth Network Access Point from Inventel (I did actually need it for another project too; I'm not quite that frivolous). Given the relative youth of the protocol and my urgent need (for my other project), this arrived without a manual of any sort and with drivers for another device entirely. But with the helpful support of the Inventel people I managed to get it working. It turned out to be running Linux, which proved immensely helpful in getting it working.

With a device like this I could attach my iPaq to the network and route TCP/IP packets over my LAN, or even the Internet (though I still don't have DNS working properly) via my ADSL line (see Figure 2). I have to admit I was surprised when my application ran without modification, though I shouldn't have been. While it helped, the battery consumption was still too high and I had to connect manually each time.

It's also wrong. While connecting to a LAN is useful, it's not what I wanted to do. My application wants to connect to a server; the LAN is used only as an intermediate step and is redundant. Bluetooth is about connecting devices to each other, not to networks (Wireless Ethernet is perfectly good for these kind of applications), so I should try to connect directly to the server. In that case, the server will also need Bluetooth connectivity, which was provided by a TDK Bluetooth USB Adapter (with the added benefit of allowing both my iPaq and Palm handhelds to synchronize wirelessly). Now that I had the hardware in place, it was time to think about how to get the software working with it.

One of the problems with Java, in fact, the primary problem, is its inability to support hardware it hasn't heard of. While there's a JSR for Bluetooth, it's still a work-in-progress, and it will be a while before we see any implementation, so we're on our own for the moment. However, Bluetooth was designed with compatibility in mind, and within the protocol are various "profiles" that define how different applications might use Bluetooth without getting involved with the whole radio thing. One of these profiles defines a standard for serial communications, so it should be possible to create a serial connection from the client to the server from Java. The iPaq defines two serial ports as working with Bluetooth: COM8 is used for outgoing connections while COM7 is used for incoming. On the PC these change to COM3 and COM4, respectively, though configurable through the TDK software. Of course, when you open a serial connection you don't normally specify a destination, so some experimentation was necessary to establish where an outgoing connection would end up. This showed an outgoing serial connection will connect to the last place a serial connection was made to, so getting it working was simply a matter of making the connection manually once, then running the software when required.

While this sounds good, there's one problem: serial connections are not part of PersonalJava. Not all PersonalJava devices are considered to have a serial port so we're reduced to the lowest common denominator. Luckily James Nord fixed this particular failing with a serial API that works on an iPaq and other Pocket PC devices, and is available free from his Web site (see Useful Links at the end of this article). He's never actually used it with Bluetooth, but helpfully pointed out that I was using an old version that didn't support the event-driven reception of data; it works perfectly with the latest version.

Testing the connection was relatively easy once I got the right version of the Serial API on my iPaq and the server; I used the same code on both. Opening the port is a matter of working through the available ports until one matches the desired port:

String port = "COM3";
portList = CommPortIdentifier.getPortIdentifiers();
while (portList.hasMoreElements()) {
portId = (CommPortIdentifier) portList.nextElement();
if (portId.getPortType() == CommPortIdentifier.PORT_SERIAL) {
if (portId.getName().startsWith(port)) {
CreateConnection();
}
}
}

Once the port has been identified, it's opened in the normal way:

serialPort = (SerialPort) portId.open("Test3", 2000);
inputStream = serialPort.getInputStream();
outputStream = serialPort.getOutputStream();
serialPort.addEventListener(this);
serialPort.notifyOnDataAvailable(true);

Setting the parameters is optional, since it won't matter if they don't match. Any parameters set will be used only in the connection between the Java application and the Bluetooth stack, but they can be set for completeness:

serialPort.setSerialPortParams(57600, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);

From this point it's just a matter of writing data as required and reading it back through events:

outputStream.write("My Message".getBytes());
outputStream.flush();

Remember to flush the buffer. The availability of data is indicated through a normal event, so the data can be read in (see Listing 1).

Now the client and server can communicate over the serial link, which is established within a second or two of the application being launched. The same protocols can be used, and the server simply needs to launch a second thread for listening and reporting serial communications (actually, it doesn't need to be a thread as the serial reading is not a blocking action). As the server was designed to cope with multiple clients, it has no problems with this and the application works.

Conclusion
Now I'm getting there: my application plays the music I like, tailored to fit my mood, and I can control it by simply turning on my iPaq and running the application; but the project isn't over yet. Some time ago I wired the whole house with speakers, as I like music wherever I go, but banks of switches are not ideal; more than once I've wandered upstairs to find that what we're watching on TV is being blasted in the bedroom. Now that I can control my music from my mobile, I need to start controlling the sound. In Part 3 I'll be adding the ability to turn speakers on and off from my application.

Useful Links

  • Pocket PC Serial Library: www.teilo.net/software
  • Desktop Serial Library: http://java.sun.com/products/javacomm/index.html
  • Bluetooth Bits: www.expansys.com
  • Digital Radio: www.psion.com
  • More Stories By Bill Ray

    Bill Ray, former editor-in-chief (and continuing distinguished contributor to) Wireless Business & Technology magazine, has been developing wireless applications for over 20 ears on just about every platform available. Heavily involved in Java since its release, he developed some of the first cryptography applications for Java and was a founder of JCP Computer Services, a company later sold to Sun Microsystems. At Swisscom he was responsible for the first Java-capable DTV set-top box, and currently holds the position of head of Enabling Software at 02, a UK network operator.

    Comments (1)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    In his opening keynote at 20th Cloud Expo, Michael Maximilien, Research Scientist, Architect, and Engineer at IBM, will motivate why realizing the full potential of the cloud and social data requires artificial intelligence. By mixing Cloud Foundry and the rich set of Watson services, IBM's Bluemix is the best cloud operating system for enterprises today, providing rapid development and deployment of applications that can take advantage of the rich catalog of Watson services to help drive insigh...
    SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @CloudExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
    SYS-CON Events announced today that Carbonite will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Carbonite protects your entire IT footprint with the right level of protection for each workload, ensuring lower costs and dependable solutions with DoubleTake and Evault.
    SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs oft...
    DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
    SYS-CON Events announced today that Outscale will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outscale's technology makes an automated and adaptable Cloud available to businesses, supporting them in the most complex IT projects while controlling their operational aspects. You boost your IT infrastructure's reactivity, with request responses that only take a few seconds.
    As cloud adoption continues to transform business, today's global enterprises are challenged with managing a growing amount of information living outside of the data center. The rapid adoption of IoT and increasingly mobile workforce are exacerbating the problem. Ensuring secure data sharing and efficient backup poses capacity and bandwidth considerations as well as policy and regulatory compliance issues.
    SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
    The 21st International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...
    SYS-CON Events announced today that Interoute has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Interoute is the owner operator of Europe's largest network and a global cloud services platform, which encompasses over 70,000 km of lit fiber, 15 data centers, 17 virtual data centers and 33 colocation centers, with connections to 195 additional partner data centers. Our full-service Unifie...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs ofte...
    SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
    SYS-CON Events announced today that Hitachi Data Systems, a wholly owned subsidiary of Hitachi LTD., will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City. Hitachi Data Systems (HDS) will be featuring the Hitachi Content Platform (HCP) portfolio. This is the industry’s only offering that allows organizations to bring together object storage, file sync and share, cloud storage gateways, and sophisticated search and...
    Existing Big Data solutions are mainly focused on the discovery and analysis of data. The solutions are scalable and highly available but tedious when swapping in and swapping out occurs in disarray and thrashing takes place. The resolution for thrashing through machine learning algorithms and support nomenclature is through simple techniques. Organizations that have been collecting large customer data are increasingly seeing the need to use the data for swapping in and out and thrashing occurs ...
    A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
    Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
    Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo Silicon Valley Call for Papers is now open.
    DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
    SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.