teaching machines

SENG 440: Lecture 19 – Nearby

May 13, 2019 by . Filed under lectures, semester1-2019, seng440.

Dear students,

Today we write an app that allows multiple devices to play a single piano using Google’s Nearby API. One device serves as the host, running a third-party MIDI interpreter. (I use FluidSynth.) A client joins the host. The host device plays octave 4, and the client device plays octave 5.

Next lecture we will discuss writing an app that is multitouch-aware. See the TODO below for the assigned exercises.

P2Piano

Prior to this lecture, I have added the following dependencies to my module’s build.gradle:

implementation 'com.google.android.gms:play-services-nearby:16.0.0'

Following are the exercises assigned last time. We will use and discuss the solutions that you’ve submitted as we assemble our app, but I include my solutions here for reference.

  1. Write class Endpoint, which accepts as parameters a string id and a DiscoveredEndpointInfo. Its public interface should expose id, name, and toString. Both name and toString should yield the endpoint’s name—which is not the same as id. Possible solution.
    class Endpoint(val id: String, private val info: DiscoveredEndpointInfo) {
  val name
    get() = info.endpointName
  override fun toString() = name
}
  2. In HostActivity, define field payloadListener as a PayloadCallback to receive a MIDI message as an array of bytes. Issue the message with KeyboardActivity.sendMidiMessage. Possible solution.
    private val payloadListener = object : PayloadCallback() {
  override fun onPayloadReceived(id: String, payload: Payload) {
    payload.asBytes()?.let { bytes ->
      sendMidiMessage(bytes)
    }
  }

  override fun onPayloadTransferUpdate(id: String, update: PayloadTransferUpdate) {}
}
  3. In HostActivity, define field connectionListener as a ConnectionLifecycleCallback. Be prepared to explain the role of its three methods. Automatically accept any attempted connection. The user should know the status of the connection. Communicate events using toasts. Possible solution.
    private val connectionListener = object : ConnectionLifecycleCallback() {
  override fun onConnectionInitiated(id: String, info: ConnectionInfo) {
    Toast.makeText(this@HostActivity, "Connecting to ${info.endpointName}.", Toast.LENGTH_SHORT).show()
    Nearby.getConnectionsClient(this@HostActivity).acceptConnection(id, payloadListener)
  }

  override fun onConnectionResult(endpoint: String, result: ConnectionResolution) {
    when (result.status.statusCode) {
      ConnectionsStatusCodes.STATUS_OK -> {
        Toast.makeText(this@HostActivity, "Connected", Toast.LENGTH_SHORT).show()
      }
      ConnectionsStatusCodes.STATUS_CONNECTION_REJECTED -> {
        Toast.makeText(this@HostActivity, "Rejected", Toast.LENGTH_SHORT).show()
      }
      ConnectionsStatusCodes.STATUS_ERROR -> {
        Toast.makeText(this@HostActivity, "Error", Toast.LENGTH_SHORT).show()
      }
    }
  }

  override fun onDisconnected(endpoint: String) {
    Toast.makeText(this@HostActivity, "Disconnected", Toast.LENGTH_SHORT).show()
  }
}
  4. In HostActivity, define method advertise to start advertising the device as a “server.” The user should know the status of the advertising. Communicate events using toasts. Possible solution.
    private fun advertise() {
  val options = AdvertisingOptions.Builder().setStrategy(Strategy.P2P_POINT_TO_POINT).build()
  Nearby.getConnectionsClient(this).startAdvertising("Chris", "P2Piano", connectionListener, options)
    .addOnSuccessListener {
      Toast.makeText(this, "Advertising...", Toast.LENGTH_LONG).show()
    }.addOnFailureListener {
      Toast.makeText(this, "Failed to advertise...", Toast.LENGTH_LONG).show()
    }
}
  5. In JoinActivity, define field payloadListener as a PayloadCallback to do absolutely nothing. The server will not be communicating anything back to the client. Possible solution.
    private val payloadListener = object : PayloadCallback() {
  override fun onPayloadReceived(endpoint: String, payload: Payload) {}
  override fun onPayloadTransferUpdate(endpoint: String, update: PayloadTransferUpdate) {}
}
  6. In JoinActivity, define field connectionListener as a ConnectionLifecycleCallback. Be prepared to explain the role of its three methods. Automatically accept any attempted connection. The user should know the status of the connection. Communicate events using toasts. Once a connection has been establishled, retain the endpoint’s ID in the hostEndpoint field. Possible solution.
    private val connectionListener = object : ConnectionLifecycleCallback() {
  override fun onConnectionInitiated(id: String, info: ConnectionInfo) {
    Toast.makeText(this@JoinActivity, "Connecting to ${info.endpointName}.", Toast.LENGTH_SHORT).show()
    Nearby.getConnectionsClient(this@JoinActivity).acceptConnection(id, payloadListener)
  }

  override fun onConnectionResult(endpoint: String, result: ConnectionResolution) {
    when (result.status.statusCode) {
      ConnectionsStatusCodes.STATUS_OK -> {
        hostEndpoint = endpoint
        Toast.makeText(this@JoinActivity, "Connected", Toast.LENGTH_SHORT).show()
      }
      ConnectionsStatusCodes.STATUS_CONNECTION_REJECTED -> {
        Toast.makeText(this@JoinActivity, "Rejected", Toast.LENGTH_SHORT).show()
      }
      ConnectionsStatusCodes.STATUS_ERROR -> {
        Toast.makeText(this@JoinActivity, "Error", Toast.LENGTH_SHORT).show()
      }
    }
  }

  override fun onDisconnected(endpoint: String) {
    Toast.makeText(this@JoinActivity, "Disconnected", Toast.LENGTH_SHORT).show()
  }
}
  7. In JoinActivity, define method joinHost, which accepts an Endpoint. Request a connection to the specified host endpoint. The user should know the status of this request. Communicate events using toasts. Possible solution.
    private fun joinHost(endpoint: Endpoint) {
  Nearby.getConnectionsClient(this@JoinActivity)
    .requestConnection("CLIENT", endpoint.id, connectionListener)
    .addOnSuccessListener {
      Toast.makeText(this@JoinActivity, "Connected to ${endpoint.name}.", Toast.LENGTH_LONG).show()
    }
    .addOnFailureListener {
      Toast.makeText(this@JoinActivity, "Rejected by ${endpoint.name}.", Toast.LENGTH_LONG).show()
    }
}
  8. In JoinActivity, define method showEndpointChooser, which accepts a list of Endpoint and returns an ArrayAdapter<Endpoint>. The endpoints in the list are possible hosts that can be joined. The method creates an adapter for the list and then displays it an AlertDialog. When an endpoint from the adapter is clicked on, join it. When the dialog is dismissed, stop discovering new hosts. Possible solution.
    private fun showEndpointChooser(items: List<Endpoint>): ArrayAdapter<Endpoint> {
  val adapter = ArrayAdapter(this, android.R.layout.simple_list_item_1, items)

  AlertDialog.Builder(this).run {
    setTitle("Choose host...")
    setAdapter(adapter) { _, i -> joinHost(items[i]) }
    setOnDismissListener {
      Nearby.getConnectionsClient(this@JoinActivity).stopDiscovery()
    }
    show()
  }

  return adapter
}
  9. In JoinActivity, define method createDiscoverListener, which accepts a MutableList of Endpoint and an ArrayAdapter of Endpoint. It returns an EndpointDiscoveryCallback that adds newly found endpoints to the list and removes lost endpoints from the list, notifying the adapter of any changes. Possible solution.
    private fun createDiscoverListener(items: MutableList<Endpoint>, adapter: ArrayAdapter<Endpoint>): EndpointDiscoveryCallback {
  return object : EndpointDiscoveryCallback() {
    override fun onEndpointFound(id: String, info: DiscoveredEndpointInfo) {
      items.add(Endpoint(id, info))
      adapter.notifyDataSetChanged()
    }

    override fun onEndpointLost(p0: String) {
      items.removeIf { it.id == p0 }
      adapter.notifyDataSetChanged()
    }
  }
}
  10. In JoinActivity, define method discover. It creates an empty MutableList of Endpoint, pops open an endpoint chooser, and creates a listener for found and lost endpoints. It starts trying to discover hosts. The user should know the status of the discovery. Communicate events using toasts. Possible solution.
    private fun discover() {
  val items = mutableListOf<Endpoint>()
  val adapter = showEndpointChooser(items)
  val callback = createDiscoverListener(items, adapter)

  val options = DiscoveryOptions.Builder().setStrategy(Strategy.P2P_POINT_TO_POINT).build()
  Nearby.getConnectionsClient(this).startDiscovery("P2Piano", callback, options).addOnSuccessListener {
    Toast.makeText(this, "Looking for hosts...", Toast.LENGTH_LONG).show()
  }.addOnFailureListener {
    Toast.makeText(this, "Discovery failed!", Toast.LENGTH_LONG).show()
  }
}
  11. In JoinActivity, override sendMidiMessage to not only call the superclass version of the method, but also send it to the paired host, if there is one. Possible solution.
    override fun sendMidiMessage(bytes: ByteArray) {
  super.sendMidiMessage(bytes)
  hostEndpoint?.let {
    Nearby.getConnectionsClient(this).sendPayload(it, Payload.fromBytes(bytes))
  }
}

Bluetooth doesn’t work on the emulator, sadly. We will have to test this with two physical devices. My wife lent me her phone for the day. Let’s hope for no emergencies!

TODO

Next lecture we will build an app that solves the problem of choosing of which player goes first in a game. All players place a finger on the phone, and the app chooses one of them randomly. The game is inspired by the commercial app Chwazi.

We will create our own custom View to receive the touch events and draw circles under each finger. To allow a custom View to be included in an XML layout, we must provide certain constructors that will get invoked by the layout inflater. I have added these constructors with some annotation magic in the following PickerView class:

class PickerView @JvmOverloads constructor(context: Context, attributes: AttributeSet? = null, style: Int = 0) : View(context, attributes, style) {
  private var pointerIdToPosition = HashMap<Int, PointF>()
  private var pickedId: Int = -1
  private var fingerprint: Drawable

  private val unpickedPaint: Paint
  private val pickedPaint: Paint
  private val prefs: SharedPreferences

  init {
    unpickedPaint = Paint().apply {
      strokeWidth = 10f
      color = Color.WHITE
      this.style = Paint.Style.STROKE
    }

    pickedPaint = Paint(unpickedPaint).apply {
      color = Color.MAGENTA
      strokeWidth = 20f
    }

    prefs = PreferenceManager.getDefaultSharedPreferences(context)
    fingerprint = resources.getDrawable(R.drawable.fingerprint, null)
  }
}

We will have two activities. The MainActivity shows the PickerView fullscreen and has this layout:

<?xml version="1.0" encoding="utf-8"?>
<org.twodee.lots.PickerView
  xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:tools="http://schemas.android.com/tools"
  android:id="@+id/picker_view"
  android:layout_width="match_parent"
  android:layout_height="match_parent"
  android:background="@android:color/black"
  tools:context=".MainActivity" />

The SettingsActivity will pop up when the user clicks on a gear icon in the action bar and that has the following layout:

<?xml version="1.0" encoding="utf-8"?>
<fragment
  xmlns:android="http://schemas.android.com/apk/res/android"
  android:id="@+id/settings_fragment"
  android:name="org.twodee.lots.SettingsFragment"
  android:layout_width="match_parent"
  android:layout_height="match_parent" />

The exercises that you will collectively complete are listed below. Check your email for your assigned exercise and a link to submit your solution. Become an expert on your particular corner of the app, investigating background material as needed. Build on top of the activity and others’ code as you complete your exercise. No solution should be very long, but some have more overhead like curly braces and method signatures than others.

  1. Declare a PreferenceScreen resource in xml/preferences.xml to present a single EditText preference for the delay in seconds from the last down or up event until one of the active pointers is randomly picked. Make the preference a decimal number with key pickDelay. Give it a title, a summary, a default of 3, and an icon of @drawable/timer.
  2. Define class SettingsFragment as a subclass of PreferenceFragmentCompat that shows the app’s PreferenceScreen.
  3. Define field pickTask in PickerView to be a Runnable that sets pickedId to a randomly chosen key from pointerIdToPosition. Also, declare field onFirstDown in PickerView to be a lambda that accepts no parameters and returns nothing. Define it to do nothing, but allow clients to redefine it to trigger some action when a first touch event occurs.
  4. Write method schedulePick that invalidates pickedId and unschedules pickTask from the inherited handler. If there are any entries left in the positions map, schedule pickTask to execute after the pick delay, which you can access via prefs. Watch your units.
  5. Write method handleDowns in PickerView to accept a MotionEvent parameter. Call onFirstDown when the first touch occurs. When any down event occurs, first or otherwise, do two things:
    • Reschedule the picking.
    • Map the active pointer to position (0, 0)—but only if the map doesn’t already have an entry for the pointer.
  6. Write method handlePositions in PickerView to accept a MotionEvent parameter. Iterate through all the currently tracked pointers and update the positions map.
  7. Write method handleUps in PickerView to accept a MotionEvent parameter. When any up event occurs, first or otherwise, do two things:
    • Reschedule the picking.
    • Remove any entry for the active pointer from the positions map.
  8. Write method onTouchEvent in PickerView to handle any down events, update the tracked pointer positions, and handle any update events. Also, so that the UI reflects the pointer state, force the view to be redrawn.
  9. Write method drawFingerprint in PickerView to accept a Canvas parameter. Draw fingerprint in the center of the view, ensuring that it fits within the bounds. Assume that the fingerprint is square.
  10. Write method onDraw in PickerView to accept a Canvas parameter. If there are no currently tracked pointers, draw the fingerprint as a cue for the user to press the screen. Otherwise, draw circles around each pointer. Use unpickedPaint by default, but if a pointer is the picked pointer, use pickedPaint.
  11. Write method enableFullscreen in MainActivity to put the app into fullscreen immersive mode. Ensure that a stable layout is used—even when the navigation controls and status bar are visible.

See you next time!

Sincerely,

P.S. It’s time for a haiku!

Bluetooth’s a weird name
But it got more votes than Flirt
“Close, but not touching”

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *