August 5, 2015

Android Wear – Beyond Notifications Part 1

Written by

For the more technical readers, Senior Software Engineer Gary Chang takes us beyond notifications with his ‘talk-to-text’ Android Wear reminder app…

Getting a notification on an Android Wear device is a very natural use case and the programming for it can be almost trivial – the Android operating system sends Google Cloud Messaging push notifications to any Wearable devices attached to your phone, with no extra programming effort.

In this series of articles, I want to cover some programming aspects of use cases that go beyond notifications, in particular:

  • Wear Data and Message APIs to interact between phone and Wear device
  • WearableListenerService to act on incoming communications asynchronously
  • Handling connection issues between the two devices
  • Wear and phone broadcast messaging to interact between background service and activities
  • Wear Speech to text recognition
  • WearableList to highlight active row on Wear device
  • Wear swipe actions (eg. delete) using GridViewPager

To illustrate each of these topics, I have implemented a simple Android Wear app that works in conjunction with a phone or tablet, called SpeeqNote. Here’s a short story to illustrate SpeeqNote in action:

  • You’re out and about on the streets with an Android Wear watch on your wrist and an Android phone in your back pocket.
  • Suddenly, you remember you have to pick up a present for your mum.
  • You tap on your watch, speak into it and dictate your reminder.
  • SpeeqNote uses the Google speech recognition feature on the watch to save your reminder as text on your phone.
  • Since you don’t want to disappoint your mum on her birthday, you marked the note as urgent so it appears at the top of the list by tapping on the coloured star beforehand.
  • While you’re glancing at your watch, you check the rest of your reminders.
  • You’ve paid the electricity bill, so your swipe your watch to delete that reminder.
  • This note is synched with the phone so it gets removed everywhere.

All of this was done while the phone was still in your back pocket.

Screenshots of the app on the Android Wear device:

androidwear5

Screenshots of the app on the phone:

androidwear2

Requirements

To implement SpeeqNote, you’ll need Android Studio 1.2, Gradle 2.4 to compile and run the app from source code (see attachment at the end of this article), a device like a phone running Android 4.3 Jelly Bean or later, and an Android Wear device. Using an Android Wear emulator will not allow the main use case of Android Wear-based speech recognition to be demonstrated, but all the other features will be fully functional.

Ok, let’s see how SpeeqNote makes all this happen!

Wear to phone: we need to talk

Android Wear devices need to talk to their attached phones to communicate with the outside world. This is done using the Data and Message APIs. Some Wearables have built in WiFi chipsets, but not all manufacturers have enabled them (probably in the interests of maximising their limited battery life) so this article will not focus on any WiFi communications.

Why are there two different types of Wear communications – DataItem and Message?

They have two different purposes:

  • The DataItem API is used to send complex data that will be synchronised automatically with any attached Wearable devices.
  • The Message API can be used to invoke commands (Remote Procedure Calls) from one device to another. This is a peer-to-peer API – the phone and Wearable device can be a sender for one command and a receiver for another. Simple byte array data can also be sent this way.

There are actually other types of communications, but we’ll be focusing only on these two types in these articles.

Communications lifecycle

You can only send or receive Messages or Data items within the appropriate part of the communications lifecycle as provided by the APIs. There are callbacks for when a connection is established (onConnected), suspended (onConnectionSuspended) or failed (onConnectionFailed) that your code may be interested in.

In this app, I have wrapped up the various parts of the lifecycle within the PhoneCommsManager and WearCommsManager classes, in order to simplify and encapsulate the intricacies of sending data from one device to its paired companion. In this demo app, it is assumed there is only one Wear device paired with the phone.

PhoneCommsManager and WearCommsManager also handle the cases where you want to send data, but the connection between the phone and the watch is not currently up-and-running (Android does not always keep the bluetooth connection established, presumably to maximise battery life). In such cases when the connection is down but we want to send data, any data is queued and the sender continues on its way. When the connection has finally been established at a later point in time, any queued Message or Data items are finally sent across to the paired device.

More information on this topic is available at https://developer.android.com/training/wearables/data-layer/accessing.html

Sending a message

You can send a message string together with an optional byte array of additional data using Wear’s Message API. All messages need to be sent to a particular device (node), identified by its nodeId.

Keeping in mind the communications lifecycle handled by PhoneCommsManager and WearCommsManager, sending a message uses the code from MessageSender / WearMessageSender as follows:

private void sendMessage(final String message) {
    ResultCallback messageCallback = new ResultCallback<>() {
        public void onResult(MessageApi.SendMessageResult sendMessageResult) {
            // did we send the message successfully?
        }
    };

    ResultCallback nodeCallback = new ResultCallback<>() {
        public void onResult(NodeApi.GetConnectedNodesResult result) {
    // only continue if we have at least one connected node (device)
 	    // get nodeId of first paired device
    	    PendingResult pendingResult = 
Wearable.MessageApi.sendMessage(apiClient, nodeId, message, optionalByteData);
    pendingResult.setResultCallback(messageCallback);
        }
    };

    Wearable.NodeApi.getConnectedNodes(GoogleApiClient).setResultCallback(nodeCallback);
}

 

Reading the code backwards due to the callback nature of the code, firstly we need to determine what connected devices (nodes) there are, by calling Wearable.NodeApi.getConnectedNodes(..)

In the node call back, we assume we are only connected to one partner device. so only send to the first device in the list and ignore any other connected devices.

The code that does the actual sending is: Wearable.MessageApi.sendMessage(..)

It requires the GoogleApiClient for the low level communications, the node id of the device you are wanting to send the message to, the message string, and an optional array of bytes that you can use to add further information you want to send.

The message callback is invoked once the message has been sent, and we simply determine whether the message was sent successfully or not. If there was a need for more sophisticated error handling – such as showing an error icon on screen – then these callbacks would need to ripple up to the calling activity, but is not done so in this demo app.

In this app, WearableActivity running on the watch uses WearMessageSender to send the messages for adding a new note, deleting an existing note, or requesting the list of all notes from the phone. There are no message senders on the phone in this demo app but the framework (MessageSender) is there.

More information on this topic is available at https://developer.android.com/training/wearables/data-layer/messages.html

Receiving a message

Once again, operating within the context of the communications lifecycle described above, receiving a message is achieved by the onMessageReceived callback method:

public void onMessageReceived(MessageEvent messageEvent) {
    String path = messageEvent.getPath();
    // process this message based on the path given in the message 
    // eg. /saveNote?priority=1&detail=Some%20Message
    // or /deleteNote?id=1234
    // or /noteList
}

 

In this app, I have placed all the message receipt functionality within the MessageListenerService / WearMessageListenerService, which we’ll talk about later. There are separate message listeners that allow for delegation of the work based on the path of the message – namely:

  • AddNoteListener which adds a new note to the database
  • DeleteNoteListener which removes an existing note based on its id
  • NoteListListener which responds with a list of all notes in the database

More information on this topic is available at:

https://developer.android.com/training/wearables/data-layer/messages.html

and

https://developer.android.com/training/wearables/data-layer/events.html

Sending data

A DataItem allows for more complex data to be represented, e.g. an object tree. It allows for multiple key – value pairs using a map structure, and maps can contain other nested maps, to result in an object tree. The URI given to create the request can be used on the receiver to route the request so that it can be appropriately actioned, e.g. /noteList

You build up one or more DataMap structures, and contain it in a PutDataMapRequest

PutDataMapRequest putDataMapReq = PutDataMapRequest.create("/myUri");
DataMap dataMap = putDataMapReq.getDataMap();
dataMap.putString("myKey", "myValue");
// can add nested DataMap structures

You then Put this DataItem out into the ether. The request is not directed at any particular node. The Android Wear framework takes care of synchronising this data with any connected nodes. Another important point is that the data will only be re-sent to other connected devices if the payload has changed, so you might want to add a time varying component such as a timestamp into the payload if you want to guarantee that the information will be sent out again – at the cost of battery life. This could be useful for testing purposes, but remember to remove it once it has been tested unless it’s really needed.

public void processRequest() {
    ResultCallback callback = new ResultCallback() {
        public void onResult(Result result) {
            // was our PutDataRequest successfully sent?
        }
    };
    PutDataRequest putDataReq = putDataMapReq; // from above
    PendingResult result = 
        Wearable.DataApi.putDataItem(apiClient, putDataReq);
    result.setResultCallback(callback);
}

 

In this app, NoteListSender extends DataItemSender to send the list of notes from the phone to the watch as the database of all notes is held on the phone.

More information on this topic is available at https://developer.android.com/training/wearables/data-layer/data-items.html

Receiving data

Similar to receiving a Message, receiving a DataItem is achieved by the onDataChanged callback method:

public void onDataChanged(DataEventBuffer dataEvents) {
    // note that there may be more than one data event in the buffer
    DataEvent event = dataEvents.get(0);
    DataItem item = event.getDataItem();
    String path = item.getUri().getPath();
    DataMap dataMap = DataMapItem.fromDataItem(item).getDataMap();
    // Route this DataItem to the appropriate processor, with the given DataMap
    // extract the individual data elements by retreiving by key as you would a HashMap:
    // String value = dataMap.getString("myKey");    and so on
}

 

In this app, WearNoteListListener handles the rehydration of the DataItem back into model objects.

More information on this topic is available at https://developer.android.com/training/wearables/data-layer/data-items.html

WearableListenerService

In this app, I have implemented the Message and DataItem receivers within a background service that extends from WearableListenerService. This allows for processing (eg. launching of a new activity) based on incoming messages and change navigation flows, regardless of which activity is currently in the foreground, or even if the app has been backgrounded. Refer to MessageListenerService and WearMessageListenerService for the implementations of WearableListenerService. There is a slight increase in complexity if you need to pass on information from the ListenerService to a foreground activity, using a broadcasted Intent. See WearNoteListListener for an example of this. All of these background services need to be registered in their respective AndroidManifest.xml files.

Note that you can also put the Message and DataItem listeners on a foreground activity as long as you properly handle the communications lifecycle call backs. Do this by implementing the MessageListener or DataApi.DataListener interfaces, respectively.

More information on this topic is available at: https://developer.android.com/training/wearables/data-layer/events.html

Handling connection issues between the phone and the Wear watch

The lifecycle callbacks sometimes are not called when you expect them to be in the real world (e.g. onConnected, onConnectionSuspended, onConnectionFailed) so to be more predictable, a workaround is to make a check for connected nodes every time an activity resumes on the Wear device. This is the same call to determine any connected nodes as used for sending messages to connected nodes described above.

In this app I’ve made the check in a common base activity class (WearableActivity.onResume) so most activities get it without additional code. If there are no connected nodes, then a connection error screen appears on the watch.

To finish off the topic of device-to-device communications, note that the Wear API supports other types of information that you can send, such as Assets for sending binary blobs of data, and Channels for streaming music or movies. They build on the Data and Message concepts just presented.

More information on this topic is available at http://developer.android.com/training/wearables/data-layer/index.html

In Part 2, we continue with the remaining topics required to build an Android Wear app that goes beyond notifications.

downloadDownload SpeeqNote source code

 

Tags: , ,

Categorised in: Digital, Innovation, Mobile

This post was written by Gary Chang