SENG 440: Lecture 23 – CameraX
Dear students,
Today we will create an app called Two-Face that allows the user to take a split image on the front-facing camera. The left half and right half are taken at separate times. We will use the new CameraX API that was just announced at Google I/O.
Before I forget, here’s your final TODO:
- Write down on a quarter sheet of paper things about the class that you think I should keep doing and things I should change. I welcome constructive feedback.
Next lecture we will wrap up the course.
Two-Face
Following are the exercises assigned last time. We will use and discuss the solutions that you’ve submitted as we assemble our app, but I include my solutions here for reference.
- Write method
createPreviewUseCasethat accepts parameters for the screen resolution as aSizeand the screen aspect ratio as aRational. Return aPreviewfor the front-facing camera, a target resolution that matches the screen resolution, a target aspect ratio that matches the screen aspect ratio, and a rotation that matches the orientation ofpreviewView. Add aOnPreviewOutputUpdateListenerto the use case that does three things:Possible solution.- Link the
SurfaceTextureofpreviewViewto the preview output. - Trigger
updatePreviewTransformto set up the transformation of the preview. Pass the output’s texture size.
private fun createPreviewUseCase(screenSize: Size, screenAspectRatio: Rational): Preview { val config = PreviewConfig.Builder().run { setLensFacing(CameraX.LensFacing.FRONT) setTargetResolution(screenSize) setTargetAspectRatio(screenAspectRatio) setTargetRotation(previewView.display.rotation) build() } val previewUseCase = Preview(config) previewUseCase.setOnPreviewOutputUpdateListener { previewOutput -> previewView.surfaceTexture = previewOutput.surfaceTexture updatePreviewTransform(previewOutput.textureSize) } return previewUseCase }
- Link the
- Write method
initializeUseCasesthat uses CameraX to bind preview and capture use cases. Compute the screen size and aspect ratio ofpreviewViewusing the dimensions yielded bypreview.display.getRealMetrics. Possible solution.private fun initializeUseCases() { val metrics = DisplayMetrics().also { previewView.display.getRealMetrics(it) } val screenSize = Size(metrics.widthPixels, metrics.heightPixels) val screenAspectRatio = Rational(metrics.widthPixels, metrics.heightPixels) val previewUseCase = createPreviewUseCase(screenSize, screenAspectRatio) captureUseCase = createCaptureUseCase(screenSize, screenAspectRatio) CameraX.bindToLifecycle(this, previewUseCase, captureUseCase) }
- Write method
createCaptureUseCasethat accepts parameters for the screen resolution as aSizeand the screen aspect ratio as aRational. Return anImageCapturefor the front-facing camera, a target aspect ratio that matches the screen aspect ratio, and a rotation that matches the orientation ofpreviewView. Possible solution.private fun createCaptureUseCase(screenSize: Size, screenAspectRatio: Rational): ImageCapture { val config = ImageCaptureConfig.Builder().run { setLensFacing(CameraX.LensFacing.FRONT) setTargetAspectRatio(screenAspectRatio) setTargetRotation(previewView.display.rotation) build() } return ImageCapture(config) }
- Write method
viewToRotationthat accepts aViewparameter. Examine the view’s display rotation. Return 0, 90, 180, or 270 accordingly. Possible solution.private fun viewToRotation(view: View) = when (view.display.rotation) { Surface.ROTATION_0 -> 0 Surface.ROTATION_90 -> 90 Surface.ROTATION_180 -> 180 Surface.ROTATION_270 -> 270 else -> 0 }
- Write method
imageProxyToBitmapthat accepts anImageProxyparameter. Return the wrapped image as aBitmap. (You’ll find more reference code dealing withImagethanImageProxy. Luckily, they have the same interface.) Possible solution.private fun imageProxyToBitmap(image: ImageProxy): Bitmap { val buffer: ByteBuffer = image.planes[0].buffer val bytes = ByteArray(buffer.remaining()) buffer.get(bytes) return BitmapFactory.decodeByteArray(bytes, 0, bytes.size) }
- Write method
rotateBitmapthat accepts parameters for aBitmapand a number of degrees as anInt. Return a new version of the bitmap has been rotated about its center by the specified number of degrees. Use a version ofBitmap.createBitmapthat accepts aMatrix. Possible solution.private fun rotateBitmap(bitmap: Bitmap, degrees: Int): Bitmap { val xform = Matrix().apply { postRotate(degrees.toFloat(), bitmap.width / 2f, bitmap.height / 2f) } return Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, xform, false) }
- Write method
meldBitmapsthat accepts parameters for a leftBitmapand a rightBitmap. Return a new bitmap in which the two images appear side-by-side. Use theARGB_8888pixel format. Possible solution.private fun meldBitmaps(left: Bitmap, right: Bitmap): Bitmap { val bitmap = Bitmap.createBitmap(left.width + right.width, left.height, Bitmap.Config.ARGB_8888) val leftPixels = IntArray(left.width * left.height) left.getPixels(leftPixels, 0, left.width, 0, 0, left.width, left.height) bitmap.setPixels(leftPixels, 0, left.width, 0, 0, left.width, left.height) val rightPixels = IntArray(left.width * left.height) right.getPixels(rightPixels, 0, right.width, 0, 0, right.width, right.height) bitmap.setPixels(rightPixels, 0, right.width, left.width, 0, right.width, right.height) return bitmap }
- Write method
saveBitmapthat accepts aBitmapparameter. It stores theBitmapas a JPEG in the top-level external storage directory under the nametwoface.jpg. Possible solution.private fun saveBitmap(bitmap: Bitmap) { val outFile = File(Environment.getExternalStorageDirectory(), "twoface.jpg") val outStream = FileOutputStream(outFile) bitmap.compress(Bitmap.CompressFormat.JPEG, 80, outStream) outStream.close() }
- Write method
meldBitmapsAndSaveMaybethat melds the two bitmaps and saves the combined bitmap—but only if bothleftBitmapandrightBitmapare valid. Possible solution.private fun meldBitmapsAndSaveMaybe() { leftBitmap?.let { left -> rightBitmap?.let { right -> val bitmap = meldBitmaps(right, left) saveBitmap(bitmap) } } }
- Write method
takeLeftImagethat triggers thecaptureUseCaseto take a picture in-memory. On a successful capture, convert the captured image to a bitmap, close the image, transform it according to the rotation, and crop it to contain only the left half of the image. Update theleftBitmapfield, try melding the bitmaps and saving the result, then sync the left half of the UI. Possible solution.private fun takeLeftImage() { captureUseCase.takePicture(object : ImageCapture.OnImageCapturedListener() { override fun onCaptureSuccess(image: ImageProxy, rotationDegrees: Int) { var bitmap = imageProxyToBitmap(image) image.close() bitmap = rotateBitmap(bitmap, rotationDegrees) bitmap = Bitmap.createBitmap(bitmap, bitmap.width / 2, 0, bitmap.width / 2, bitmap.height) leftBitmap = bitmap meldBitmapsAndSaveMaybe() syncLeft() } }) }
- Write method
takeRightImagethat triggers thecaptureUseCaseto take a picture in-memory. On a successful capture, convert the captured image to a bitmap, close the image, transform it according to the rotation, and crop it to contain only the right half of the image. Update therightBitmapfield, try melding the bitmaps and saving the result, then sync the right half of the UI. Possible solution.private fun takeRightImage() { captureUseCase.takePicture(object : ImageCapture.OnImageCapturedListener() { override fun onCaptureSuccess(image: ImageProxy, rotationDegrees: Int) { var bitmap = imageProxyToBitmap(image) image.close() bitmap = rotateBitmap(bitmap, rotationDegrees) bitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.width / 2, bitmap.height) rightBitmap = bitmap meldBitmapsAndSaveMaybe() syncRight() } }) }
See you next time!
Sincerely,
P.S. It’s time for a haiku!
Cameras these days
Seems they do it all for you
Even bunny ears