teaching machines

CS 491 Lecture 15 – Working with the Camera

October 22, 2013 by . Filed under cs491 mobile, fall 2013, lectures.

Agenda

TODO

Exercise

Two things are true:

  1. Some of the best stories are told as a three-panel comic strip.
  2. We need to learn how to use the camera on a mobile device.

Let’s exploit this plurality of truth by writing an app that lets you compose a three panel photo series. For example:

An example three-panel image sequence, the like of which you’ll use your app to create.

We’ll need a layout that has three image views or three buttons laid out horizontally. On click, we want to pop up a camera interface. The resulting image is inserted in the image view that was originally clicked.

If we have time, we’ll save the images out to the filesystem so that they can be reloaded later.

iOS

I suggest the following sequence of steps to get ThreePanel up and running on iOS:

  1. Create a single view application. It’s tempting to add three UIImageViews to the storyboard, but getting them each to consume 1/3 of the view is not straightforward. I suggest adding and sizing the UIImageViews programmatically instead. Before you can do so, you need to disable autolayout for the view. (If you don’t disable autolayout on a parent view, changing the size of a child view will have no effect.) Select the View in the storyboard, go to the Attributes Inspector, and uncheck Autoresize Subviews.
  2. Add an NSMutableArray property to your view controller to hold your three image views.
  3. Ask yourself, “Where shall I create my UIImageViews?” You want each to consume 1/3 of the screen width. The first callback we get after the storyboard has been loaded is viewDidLoad, but the view’s width may not be correct at this stage. The first callback you get where we do know the size of the view is viewWillLayoutSubviews. But this method may get called many times while your app executes. Construction should only happen once. So, for construction, override viewDidLoad. I had good success with the following implementation of this method:
    construct image views array
    for i to 3
      construct imageview
      add imageview as subview to self.view
      add imageview to array
    
      -- allow clicks to be processed by view, which is not normally enabled for image views
      set imageview user interaction enabled to true
    
      -- add a tap listening method
      UITapGestureRecognizer *tapRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(didPressImage:)];
      add gesture recognizer to imageview
    end
  4. The resizing is best done in viewWillLayoutSubviews, which will be called on both initial construction and during orientation changes. The following algorithm worked for me:
    bounds = self.view.bounds
    for i to 3
      frame = CGRectMake(upperleft corner x, upperleft corner y, 1/3 * bounds.width, bounds.height)
      imageView[i].frame = frame
    end
  5. When we added the tap listeners earlier, we said we had a method didPressImage: method. Let’s add that now. It should pop open the image picker interface. We only allow images to come directly from the camera:
    - (void)didPressImage:(UITapGestureRecognizer *)recognizer {    
        UIImagePickerController *imagePickerController = [[UIImagePickerController alloc] init];
        [imagePickerController setSourceType:UIImagePickerControllerSourceTypeCamera];
        [imagePickerController setDelegate:self];
        [self presentViewController:imagePickerController animated:YES completion:nil];
    }
  6. We set our view controller as the delegate for the image picker events. That means we’ll get a callback when a picture has been taken. In order to be a legal delegate, our view controller must be a UINavigationControllerDelegate and a UIImagePickerControllerDelegate. That is, we must satisfy the interface of these delegate protocols. We can say we’re of the right type in the header file:
    @interface ViewController : UIViewController <UIImagePickerControllerDelegate, UINavigationControllerDelegate>

    We also need to implement the imagePickerController:didFinishPickingMediaWithInfo: callback:

    - (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {
        // close the picker dialog
        [picker dismissViewControllerAnimated:YES completion:nil];
    
        // pull out the image from the info dictionary
        UIImage *image = [info objectForKey:UIImagePickerControllerOriginalImage];
    
        set the clicked upon imageview's image to image
    }
  7. If you want to write an image to a file, you can do so with the following code:
    // Find a directory that only this app can write to and read from.
    NSString *dir = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
    
    // Let's store the image as $dir/image[012].png.
    NSString *path = [dir stringByAppendingPathComponent:[NSString stringWithFormat:@"image%d.png", i]];
    
    // Turn the image into a big block of JPEG bytes.
    NSData *imageAsData = UIImageJPEGRepresentation(image, 0.5);
    
    // Write the block to disk.
    [imageAsData writeToFile:path atomically:YES];
    

    Reading is similar, but reversed:

    NSData *imageAsData = [NSData dataWithContentsOfFile:path];
    if (imageAsData) {
      UIImage *image = [UIImage imageWithData:imageAsData];
      imageView.image = image;
    }

Android

I suggest the following sequence of steps to get ThreePanel up and running on Android:

  1. Android’s XML layout system makes it pretty easy to put three imageviews up that each consume 1/3 of the screen. Create a standard application, but replace the RelativeLayout that is automatically generated with a horizontal linear layout. Add three ImageViews inside of it, giving each a layout_width of 0dp (our way of saying that we want the size to be figured out by the layout system) and a layout_weight of 1 (meaning that each of three imageviews will gain the same proportion of any unconsumed space in the parent layout).
  2. Add onClick listener’s to each of the imageviews. Have them call a takePicture method that you write.
  3. In takePicture, issue an intent to have the camera pop up for picture taking:

    Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    startActivityForResult(intent, REQUEST_CODE_OF_YOUR_CHOOSING);
  4. To have the camera store the image for you in your app’s pictures directory, add an extra to your intent:
    File dir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
    File imageFile = new File(dir, "image" + i + ".jpg");
    intent.putExtra(MediaStore.EXTRA_OUTPUT, Uri.fromFile(imageFile));
  5. To be notified when the camera is done with it’s work, we must override the onActivityResult callback:
    @Override
    protected void onActivityResult(int requestCode,
                                    int resultCode,
                                     Intent data) {
      if (requestCode >= 100 && requestCode <= 102) {
        if (resultCode == RESULT_OK) {
        } else {
          Log.d("FOO", "photo canceled");
        }
      } else {
        super.onActivityResult(requestCode, resultCode, data);
      }
    } 
  6. If the result is okay, we can pull the image back off of disk and insert it into the clicked-upon imageview. I ran into some memory issues when I did this (the camera takes high-resolution pictures!), so I found it necessary to scale the image down a bit. The Android documentation offers some pseudocode to scale an image to fit a view just perfectly:
    // Get the dimensions of the View
    int targetWidth = imageViews[i].getWidth();
    int targetHeight = imageViews[i].getHeight();
    
    // Get the dimensions of the bitmap
    BitmapFactory.Options options = new BitmapFactory.Options();
    options.inJustDecodeBounds = true;
    BitmapFactory.decodeFile(imageFile.getAbsolutePath(), options);
    int sourceWidth = options.outWidth;
    int sourceHeight = options.outHeight;
    
    // Determine how much to scale down the image
    int scaleFactor = Math.min(sourceWidth / targetWidth, sourceHeight / targetHeight);
    
    // Decode the image file into a Bitmap sized to fill the View
    options.inJustDecodeBounds = false;
    options.inSampleSize = scaleFactor;
    options.inPurgeable = true;
    
    Bitmap bitmap = BitmapFactory.decodeFile(imageFile.getAbsolutePath(), options);
    
  7. If you execute the above code on the main thread, your UI will be pretty sluggish. I’d suggest throwing it in an AsyncTask.

Haiku

PicMe started well.
I’d see photos from my day.
Then it turned meta.