» SENG 440: Lecture 21 – Panning and Zooming

SENG 440: Lecture 21 – Panning and Zooming

May 20, 2019 by Chris Johnson. Filed under lectures, semester1-2019, seng440.

Dear students,

Today we will examine two common gestures that occur on mobile platforms: pinch-and-zoom and panning. We won’t create a full app, just a custom view that displays an image and supports these two gestures. First we’ll implement it by just adjusting the bounding rectangle containing the image. Once that works, we’ll switch transform the image with matrices.

Next lecture we will develop an anagram solving app that uses speech recognition to receive the player’s answers. See the TODO below for the assigned exercises.

Consider this extra credit opportunity:

On Friday, 24 May at 1 PM, Chris Johnson will presenting a seminar entitled Computational Making in JE445. Attend and write down a response on a quarter sheet for an extra 0.5 Pakipaki.

ZoomPan

Following are the exercises assigned last time. We will use and discuss the solutions that you’ve submitted as we assemble our app, but I include my solutions here for reference.

Define field panListener to be a SimpleOnGestureListener. When a scroll event happens, it shifts the bounding box by the scrolled distance and forces the view to be redrawn. Possible solution.

private val panListener = object : GestureDetector.SimpleOnGestureListener() {
  override fun onDown(e: MotionEvent): Boolean {
    return true
  }

  override fun onScroll(e1: MotionEvent, e2: MotionEvent, distanceX: Float, distanceY: Float): Boolean {
    bounds.offset(-distanceX, -distanceY)
    invalidate()
    return true
  }
}

Define field zoomListener to be a SimpleOnScaleGestureListener. When a scale event happens, scale the bounding box. Bonus points if you clamp the scale from becoming too small. Tall poppy points if you scale about fingers’ centroid rather than (0, 0). Possible solution.

private val zoomListener = object : ScaleGestureDetector.SimpleOnScaleGestureListener() {
  override fun onScale(detector: ScaleGestureDetector): Boolean {
    bounds.left = ((bounds.left - detector.focusX) * detector.scaleFactor) + detector.focusX
    bounds.right = ((bounds.right - detector.focusX) * detector.scaleFactor) + detector.focusX
    bounds.top = ((bounds.top - detector.focusY) * detector.scaleFactor) + detector.focusY
    bounds.bottom = ((bounds.bottom - detector.focusY) * detector.scaleFactor) + detector.focusY

    invalidate()

    return true
  }
}

Investigate gesture detectors in Android. Define fields panDetector and zoomDetector to be detectors for the two listeners defined in the previous steps. Possible solution.
```
private val panDetector = GestureDetector(context, panListener)
private val zoomDetector = ScaleGestureDetector(context, zoomListener)
```

Define method fillWidth to set the bounds such that the image fills the view’s width. Set the height to maintain the image’s aspect ratio. Possible solution.

private fun fillWidth() {
  val imageAspect = image.width / image.height.toFloat()

  val fitWidth = width.toFloat()
  val fitHeight = width / imageAspect

  val centerX = width * 0.5f
  val centerY = height * 0.5f

  bounds.left = centerX - fitWidth * 0.5f
  bounds.right = centerX + fitWidth * 0.5f
  bounds.top = centerY - fitHeight * 0.5f
  bounds.bottom = centerY + fitHeight * 0.5f
}

Define method fillHeight to set the bounds such that the image fills the view’s height. Set the width to maintain the image’s aspect ratio. Possible solution.

private fun fillHeight() {
  val imageAspect = image.width / image.height.toFloat()

  val fitHeight = height.toFloat()
  val fitWidth = height * imageAspect

  val centerX = width * 0.5f
  val centerY = height * 0.5f

  bounds.left = centerX - fitWidth * 0.5f
  bounds.right = centerX + fitWidth * 0.5f
  bounds.top = centerY - fitHeight * 0.5f
  bounds.bottom = centerY + fitHeight * 0.5f
}

Define method onSizeChanged to set the bounds to fill the view as much possible—filling either the width or height. Bonus points if you figure out what criteria can fit the image to the view without cropping any of it. Possible solution.

override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
  super.onSizeChanged(w, h, oldw, oldh)

  val imageAspect = image.width / image.height.toFloat()
  val frameAspect = w / h.toFloat()

  if (imageAspect < frameAspect) {
    fillHeight()
  } else {
    fillWidth()
  }
}

Define method onDraw to draw the bitmap to fill the current bounds. Possible solution.

override fun onDraw(canvas: Canvas) {
  canvas.drawBitmap(image, null, bounds, null)
}

Define method onTouchEvent to let the detectors process the event. Possible solution.

override fun onTouchEvent(event: MotionEvent): Boolean {
  zoomDetector.onTouchEvent(event)
  panDetector.onTouchEvent(event)
  return true
}

We are manually tuning the bounding rectangle parameters. Many graphics systems employ matrices to manage these transformations. Let’s see what this app would look like using matrices. First, we need a running transformation, which we declare as an instance variable:

private val xform: Matrix = Matrix()

The panListener gets this onScroll:

override fun onScroll(e1: MotionEvent, e2: MotionEvent, distanceX: Float, distanceY: Float): Boolean {
  xform.postTranslate(-distanceX, -distanceY)
  invalidate()
  return true
}

The zoomListener gets this onScale:

override fun onScale(detector: ScaleGestureDetector): Boolean {
  xform.postTranslate(-detector.focusX, -detector.focusY)
  xform.postScale(detector.scaleFactor, detector.scaleFactor)
  xform.postTranslate(detector.focusX, detector.focusY)

  invalidate()
  return true
}

We want to scale about the centroid of the fingers, so we first subtract off the centroid to make it behave like the origin. Only then do we scale. Because we shifted the centroid, we must also unshift.

Method onSizeChanged must scale and center the image in the viewport. We first shift the image to its center, then we scale it, and then we shift the center to the middle of the viewport.

override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
  super.onSizeChanged(w, h, oldw, oldh)

  val imageAspect = image.width / image.height.toFloat()
  val frameAspect = w / h.toFloat()

  xform.reset()

  val scale = if (imageAspect < frameAspect) {
    h / image.height.toFloat()
  } else {
    w / image.width.toFloat()
  }

  xform.postTranslate(-0.5f * image.width, -0.5f * image.height)
  xform.postScale(scale, scale)
  xform.postTranslate(0.5f * w, 0.5f * h)
}

Method onDraw uses a different version of drawBitmap:

override fun onDraw(canvas: Canvas) {
  canvas.drawBitmap(image, xform, null)
}

Which version is better? If you are familiar with matrices and their transform semantics, they are generally more versatile—watch our bounding box approach fail when we add rotation. Also, they usually yield more compact code. But for simple needs, matrices certainly aren’t necessary.

TODO

Next lecture we will create an app called Recog that presents anagrams for the user to unscramble. But instead of typing, the user will speak the answer. We’ll use Android’s speech recognition facilities to make this happen. Please read up on speech recognition.

Our main activity will present only a TextView to show the current anagram. We’ll start with this code:

class MainActivity : PermittedActivity() {
  private lateinit var wordLabel: TextView
  private lateinit var recognizer: SpeechRecognizer
  private lateinit var recognizeIntent: Intent

  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_main)
    wordLabel = findViewById(R.id.wordLabel)

    generateRandomWord()
  }
}

Random words will be fetched from the Words API web service. They start charging after 2500 requests in a day, so I will not be sharing my key with you. Sorry. Feel free to make your own. Just keep it out of Git.

I have overhauled some code that we wrote earlier in the semester to ease the consumption of their API. Function parameterUrl encodes and appends parameters to a URL for a GET request:

fun parameterizeUrl(url: String, parameters: Map<String, String>): URL {
  val builder = Uri.parse(url).buildUpon()
  parameters.forEach { (key, value) ->
    builder.appendQueryParameter(key, value)
  }
  val uri = builder.build()
  return URL(uri.toString())
}

Function getJson retrieves the JSON from a service request, sending along any extra headers required by the service:

fun getJson(url: URL, headers: Map<String, String>): JSONObject {
  val connection = url.openConnection() as HttpsURLConnection

  headers.forEach { (key, value) ->
    connection.setRequestProperty(key, value)
  }

  try {
    val json = BufferedInputStream(connection.inputStream).readBytes().toString(Charset.defaultCharset())
    return JSONObject(json)
  } finally {
    connection.disconnect()
  }
}

The exercises that you will collectively complete are listed below. Check your email for your assigned exercise and a link to submit your solution. Become an expert on your particular corner of the app, investigating background material as needed. Build on top of the activity and others’ code as you complete your exercise. No solution should be very long, but some have more overhead like curly braces and method signatures than others.

Write extension function anagram for String. It generates a new String that is a random shuffling of the letters of the original String. Ensure that the shuffling isn’t identical to the original String.
Write extension function hasSameLetters for String that accepts another String as a parameter. It returns true if the two Strings have the same letters. For example, "these".hasSameLetters("sheet") returns true. We need this method because some anagrams may have multiple correct answers, and we want to accept any of them, not just the one we have in mind. For example, reab could unscramble to either bear or bare. Both have the same pronunciation. So it is with break and brake. Can you think of others?
Define class RandomWordFetcher to extend AsyncTask and accept a MainActivity, a host string, and a key string as constructor parameters. In the background, fetch the JSON from the Words API endpoint at https://wordsapiv1.p.mashape.com/words/, with the query parameters random=true, frequencyMin=4, and letters=5. Send also the headers X-RapidAPI-Host and X-RapidAPI-Key with the specified strings. The resulting JSON will be formatted according to Words API. In the main thread, assign the randomly generated word to the word property of MainActivity.
Define method generateRandomWord in MainActivity to start up a new RandomWordFetcher task. Pass string resources with IDs words_api_host and words_api_key.
Define property word in MainActivity to show the word in wordLabel and listen to the user.
Define method listen to start a new speech recognition activity. Use request code 440.
Define method onActivityResult to respond to the results of listening. Grab the candidate utterances and check to see if any of the candidates is a correct unscrambling.
Define method checkCandidates to accept an ArrayList of String. If any element of the list has the same letters as the current word, generate a new word. Otherwise, pop up a toast advising the player to try again and listen to the user for another attempt.
Define field recognitionListener to be an instance of RecognitionListener. If an error occurs, pop up a toast warning. If no error occurs, grab the candidate utterances and check to see if any of the candidates is a correct unscrambling.
Define method initializeSansDialog to create a new SpeechRecognizer that calls back to recognitionListener. Once it’s constructured, generate a new random word.

See you next time!

Sincerely,

P.S. It’s time for a haiku!

Lazy programmers
Circle extends Rectangle
That’s cutting corners