SENG 440: Lecture 21 – Panning and Zooming
Dear students,
Today we will examine two common gestures that occur on mobile platforms: pinch-and-zoom and panning. We won’t create a full app, just a custom view that displays an image and supports these two gestures. First we’ll implement it by just adjusting the bounding rectangle containing the image. Once that works, we’ll switch transform the image with matrices.
Next lecture we will develop an anagram solving app that uses speech recognition to receive the player’s answers. See the TODO below for the assigned exercises.
Consider this extra credit opportunity:
On Friday, 24 May at 1 PM, Chris Johnson will presenting a seminar entitled Computational Making in JE445. Attend and write down a response on a quarter sheet for an extra 0.5 Pakipaki.
ZoomPan
Following are the exercises assigned last time. We will use and discuss the solutions that you’ve submitted as we assemble our app, but I include my solutions here for reference.
- Define field
panListenerto be aSimpleOnGestureListener. When a scroll event happens, it shifts the bounding box by the scrolled distance and forces the view to be redrawn. Possible solution.private val panListener = object : GestureDetector.SimpleOnGestureListener() { override fun onDown(e: MotionEvent): Boolean { return true } override fun onScroll(e1: MotionEvent, e2: MotionEvent, distanceX: Float, distanceY: Float): Boolean { bounds.offset(-distanceX, -distanceY) invalidate() return true } }
- Define field
zoomListenerto be aSimpleOnScaleGestureListener. When a scale event happens, scale the bounding box. Bonus points if you clamp the scale from becoming too small. Tall poppy points if you scale about fingers’ centroid rather than (0, 0). Possible solution.private val zoomListener = object : ScaleGestureDetector.SimpleOnScaleGestureListener() { override fun onScale(detector: ScaleGestureDetector): Boolean { bounds.left = ((bounds.left - detector.focusX) * detector.scaleFactor) + detector.focusX bounds.right = ((bounds.right - detector.focusX) * detector.scaleFactor) + detector.focusX bounds.top = ((bounds.top - detector.focusY) * detector.scaleFactor) + detector.focusY bounds.bottom = ((bounds.bottom - detector.focusY) * detector.scaleFactor) + detector.focusY invalidate() return true } }
- Investigate gesture detectors in Android. Define fields
panDetectorandzoomDetectorto be detectors for the two listeners defined in the previous steps. Possible solution.private val panDetector = GestureDetector(context, panListener) private val zoomDetector = ScaleGestureDetector(context, zoomListener)
- Define method
fillWidthto set the bounds such that the image fills the view’s width. Set the height to maintain the image’s aspect ratio. Possible solution.private fun fillWidth() { val imageAspect = image.width / image.height.toFloat() val fitWidth = width.toFloat() val fitHeight = width / imageAspect val centerX = width * 0.5f val centerY = height * 0.5f bounds.left = centerX - fitWidth * 0.5f bounds.right = centerX + fitWidth * 0.5f bounds.top = centerY - fitHeight * 0.5f bounds.bottom = centerY + fitHeight * 0.5f }
- Define method
fillHeightto set the bounds such that the image fills the view’s height. Set the width to maintain the image’s aspect ratio. Possible solution.private fun fillHeight() { val imageAspect = image.width / image.height.toFloat() val fitHeight = height.toFloat() val fitWidth = height * imageAspect val centerX = width * 0.5f val centerY = height * 0.5f bounds.left = centerX - fitWidth * 0.5f bounds.right = centerX + fitWidth * 0.5f bounds.top = centerY - fitHeight * 0.5f bounds.bottom = centerY + fitHeight * 0.5f }
- Define method
onSizeChangedto set the bounds to fill the view as much possible—filling either the width or height. Bonus points if you figure out what criteria can fit the image to the view without cropping any of it. Possible solution.override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) { super.onSizeChanged(w, h, oldw, oldh) val imageAspect = image.width / image.height.toFloat() val frameAspect = w / h.toFloat() if (imageAspect < frameAspect) { fillHeight() } else { fillWidth() } }
- Define method
onDrawto draw the bitmap to fill the current bounds. Possible solution.override fun onDraw(canvas: Canvas) { canvas.drawBitmap(image, null, bounds, null) }
- Define method
onTouchEventto let the detectors process the event. Possible solution.override fun onTouchEvent(event: MotionEvent): Boolean { zoomDetector.onTouchEvent(event) panDetector.onTouchEvent(event) return true }
We are manually tuning the bounding rectangle parameters. Many graphics systems employ matrices to manage these transformations. Let’s see what this app would look like using matrices. First, we need a running transformation, which we declare as an instance variable:
private val xform: Matrix = Matrix()
The panListener gets this onScroll:
override fun onScroll(e1: MotionEvent, e2: MotionEvent, distanceX: Float, distanceY: Float): Boolean {
xform.postTranslate(-distanceX, -distanceY)
invalidate()
return true
}
The zoomListener gets this onScale:
override fun onScale(detector: ScaleGestureDetector): Boolean {
xform.postTranslate(-detector.focusX, -detector.focusY)
xform.postScale(detector.scaleFactor, detector.scaleFactor)
xform.postTranslate(detector.focusX, detector.focusY)
invalidate()
return true
}
We want to scale about the centroid of the fingers, so we first subtract off the centroid to make it behave like the origin. Only then do we scale. Because we shifted the centroid, we must also unshift.
Method onSizeChanged must scale and center the image in the viewport. We first shift the image to its center, then we scale it, and then we shift the center to the middle of the viewport.
override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
super.onSizeChanged(w, h, oldw, oldh)
val imageAspect = image.width / image.height.toFloat()
val frameAspect = w / h.toFloat()
xform.reset()
val scale = if (imageAspect < frameAspect) {
h / image.height.toFloat()
} else {
w / image.width.toFloat()
}
xform.postTranslate(-0.5f * image.width, -0.5f * image.height)
xform.postScale(scale, scale)
xform.postTranslate(0.5f * w, 0.5f * h)
}
Method onDraw uses a different version of drawBitmap:
override fun onDraw(canvas: Canvas) {
canvas.drawBitmap(image, xform, null)
}
Which version is better? If you are familiar with matrices and their transform semantics, they are generally more versatile—watch our bounding box approach fail when we add rotation. Also, they usually yield more compact code. But for simple needs, matrices certainly aren’t necessary.
TODO
Next lecture we will create an app called Recog that presents anagrams for the user to unscramble. But instead of typing, the user will speak the answer. We’ll use Android’s speech recognition facilities to make this happen. Please read up on speech recognition.
Our main activity will present only a TextView to show the current anagram. We’ll start with this code:
class MainActivity : PermittedActivity() {
private lateinit var wordLabel: TextView
private lateinit var recognizer: SpeechRecognizer
private lateinit var recognizeIntent: Intent
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
wordLabel = findViewById(R.id.wordLabel)
generateRandomWord()
}
}
Random words will be fetched from the Words API web service. They start charging after 2500 requests in a day, so I will not be sharing my key with you. Sorry. Feel free to make your own. Just keep it out of Git.
I have overhauled some code that we wrote earlier in the semester to ease the consumption of their API. Function parameterUrl encodes and appends parameters to a URL for a GET request:
fun parameterizeUrl(url: String, parameters: Map<String, String>): URL {
val builder = Uri.parse(url).buildUpon()
parameters.forEach { (key, value) ->
builder.appendQueryParameter(key, value)
}
val uri = builder.build()
return URL(uri.toString())
}
Function getJson retrieves the JSON from a service request, sending along any extra headers required by the service:
fun getJson(url: URL, headers: Map<String, String>): JSONObject {
val connection = url.openConnection() as HttpsURLConnection
headers.forEach { (key, value) ->
connection.setRequestProperty(key, value)
}
try {
val json = BufferedInputStream(connection.inputStream).readBytes().toString(Charset.defaultCharset())
return JSONObject(json)
} finally {
connection.disconnect()
}
}
The exercises that you will collectively complete are listed below. Check your email for your assigned exercise and a link to submit your solution. Become an expert on your particular corner of the app, investigating background material as needed. Build on top of the activity and others’ code as you complete your exercise. No solution should be very long, but some have more overhead like curly braces and method signatures than others.
- Write extension function
anagramforString. It generates a newStringthat is a random shuffling of the letters of the originalString. Ensure that the shuffling isn’t identical to the originalString. - Write extension function
hasSameLettersforStringthat accepts anotherStringas a parameter. It returns true if the twoStrings have the same letters. For example,"these".hasSameLetters("sheet")returns true. We need this method because some anagrams may have multiple correct answers, and we want to accept any of them, not just the one we have in mind. For example, reab could unscramble to either bear or bare. Both have the same pronunciation. So it is with break and brake. Can you think of others? - Define class
RandomWordFetcherto extendAsyncTaskand accept aMainActivity, a host string, and a key string as constructor parameters. In the background, fetch the JSON from the Words API endpoint athttps://wordsapiv1.p.mashape.com/words/, with the query parametersrandom=true,frequencyMin=4, andletters=5. Send also the headersX-RapidAPI-HostandX-RapidAPI-Keywith the specified strings. The resulting JSON will be formatted according to Words API. In the main thread, assign the randomly generated word to thewordproperty ofMainActivity. - Define method
generateRandomWordinMainActivityto start up a newRandomWordFetchertask. Pass string resources with IDswords_api_hostandwords_api_key. - Define property
wordinMainActivityto show the word inwordLabelandlistento the user. - Define method
listento start a new speech recognition activity. Use request code 440. - Define method
onActivityResultto respond to the results of listening. Grab the candidate utterances and check to see if any of the candidates is a correct unscrambling. - Define method
checkCandidatesto accept anArrayListofString. If any element of the list has the same letters as the current word, generate a new word. Otherwise, pop up a toast advising the player to try again and listen to the user for another attempt. - Define field
recognitionListenerto be an instance ofRecognitionListener. If an error occurs, pop up a toast warning. If no error occurs, grab the candidate utterances and check to see if any of the candidates is a correct unscrambling. - Define method
initializeSansDialogto create a newSpeechRecognizerthat calls back torecognitionListener. Once it’s constructured, generate a new random word.
See you next time!
P.S. It’s time for a haiku!
Lazy programmersCircle extends Rectangle
That’s cutting corners