Today I looked into going in reverse: can I call Unity methods from Android? It turns out to be pretty simple. I first made a little controller script that I attached to a Sphere game object named Ball:

using UnityEngine;
using System.Collections;

public class BallController : MonoBehaviour {
  private const float JUMP = 1.0f;

  public void Left() {
    transform.Translate(-JUMP, 0.0f, 0.0f);
  }

  void Right() {
    transform.Translate(JUMP, 0.0f, 0.0f);
  }

  void Up() {
    transform.Translate(0.0f, JUMP, 0.0f);
  }

  void Down() {
    transform.Translate(0.0f, -JUMP, 0.0f);
  }
}

In my custom Android Activity, I can invoke one of these methods like so:

UnityPlayer.UnitySendMessage("Ball", "Left", "");

The first parameter is the game object whose method I want to call. The second is the method I want to call on that game object. The last parameter in that call is for parameters to the method. There are none for Left, so I left it blank. That’s all there was to it.

The resulting .class file(s) should be compressed into a .jar file and placed in the Assets->Plugins->Android folder. Since the manifest dictates which activity to launch it is also necessary to create a new AndroidManifest.xml. The AndroidManifest.xml file should also be placed in the Assets->Plugins->Android folder.

It’s less clear how to package up resources, assets, and libs, but through forum-scrounging and trial and error, I’ve learned that I can simply mirror res, assets, and libs in Assets/Plugins/Android. With those in place, Unity will merge its own resources and other files with mine.

One drawback to this merge is that my R.class file in my *.jar file is not brought over. (Even if I include it, Unity appears to ignore it.) Instead, a brand new R class will be generated. That means my R.*.* identifiers will not work. The workaround is to query for the IDs dynamically, like so:

int textViewID = getResources().getIdentifier("caption", "id", getPackageName());
TextView textView = (TextView) findViewById(textViewID);

// I'd rather do it this way:
// TextView textView = (TextView) findViewById(R.id.caption);

The code above assumes I’m in my custom Activity class.

Custom Layouts

One of the resources I wanted to bring over was a custom overlay that dropped on top of the Unity game screen. I created a simple FrameLayout with a ToggleButton on it:

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout
  xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="match_parent"
  android:layout_height="match_parent">
  <ToggleButton
    android:id="@+id/toggleButton"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_gravity="center_horizontal|bottom" />
</FrameLayout>

Getting this to appear on top of the Unity game screen turned out to be pretty easy. I inflate the XML (overlay.xml) and addContentView in my onCreate method:

int layoutID = getResources().getIdentifier("overlay", "layout", getPackageName());
int buttonID = getResources().getIdentifier("toggleButton", "id", getPackageName());

FrameLayout layout = (FrameLayout) LayoutInflater.from(this).inflate(layoutID, null);
button = (ToggleButton) layout.findViewById(buttonID);

addContentView(layout, new LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));

The ToggleButton appeared beautifully at the bottom of the screen. It toggled and I rejoiced.

Custom Strings

Next up I wanted to draw custom strings depending on the locale. I added the following to res/values/strings.xml:

<string name="line0">ay</string>
<string name="line1">bee</string>
<string name="line2">cee</string>
<string name="line3">dee</string>

I made Spanish versions in res/values-es/strings.xml:

<string name="line0">ah</string>
<string name="line1">bay</string>
<string name="line2">say</string>
<string name="line3">day</string>

Pulling them out sequentially on toggles was pretty straightforward. Here’s the complete Activity:

public class MainActivity extends UnityPlayerActivity {
  private int n;
  private ToggleButton button;

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    int layoutID = getResources().getIdentifier("overlay", "layout", getPackageName());
    int buttonID = getResources().getIdentifier("toggleButton", "id", getPackageName());

    FrameLayout layout = (FrameLayout) LayoutInflater.from(this).inflate(layoutID, null);
    button = (ToggleButton) layout.findViewById(buttonID);

    addContentView(layout, new LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));

    button.setOnCheckedChangeListener(new OnCheckedChangeListener() {
      @Override
      public void onCheckedChanged(CompoundButton buttonView,
                                   boolean isChecked) {
        Toast.makeText(MainActivity.this, "Checked: " + isChecked + " -> " + getNext(), Toast.LENGTH_SHORT).show();
      }
    });

    n = 0;
  }

  public String get(int n) {
    return getString(getResources().getIdentifier("line" + n, "string", getPackageName()));
  }

  public String getNext() {
    String s = get(n);
    n++;
    if (n > 3) {
      n = 0;
    }
    return s;
  }
}

Switching locales dynamically works just fine. I start reading from the other locale’s resource directory on the next toggle after a locale switch.

<project default="build">
  <!-- Set this to your top-level Unity project directory! -->
  <property
    name="project.dir"
    value="D:/rezcycle" />

  <!-- All Android files will be placed inside Assets/Plugins/Android. No 
    need to touch this. -->
  <property
    name="assets.dir"
    value="${project.dir}/Assets/Plugins/Android" />

  <target name="build">
    <!-- Make the assets directory in our project. -->
    <mkdir dir="${assets.dir}" />

    <!-- Copy over the manifest. -->
    <copy
      file="AndroidManifest.xml"
      todir="${assets.dir}" />

    <!-- Copy over the raw files in res/ and assets/. -->
    <copy todir="${assets.dir}">
      <fileset
        dir="."
        includes="res/**" />
      <fileset
        dir="."
        includes="assets/**" />
    </copy>

    <!-- Produce a JAR file of all your pre-compiled classes. Don't bring 
      R stuff. -->
    <jar destfile="${assets.dir}/custom_android.jar">
      <fileset
        dir="bin/classes"
        excludes="**/R.class,**/R$*.class" />
    </jar>
  </target>
</project>

I drop this in to my top-level Android project directory in Eclipse, naming it build.xml. Eclipse knows how to run Ant files, so I simply run as “Ant Build”.

public RecognizerTask(File sphinxDirectory,
                      String dictionary,
                      String lm) {
  pocketsphinx.setLogfile(sphinxDirectory.getAbsolutePath() + "/pocketsphinx.log");

  Config c = new Config();
  c.setString("-hmm", sphinxDirectory + "/hmm/en_US/hub4wsj_sc_8k");
  c.setString("-dict", sphinxDirectory + "/lm/en_US/" + dictionary);
  c.setString("-lm", sphinxDirectory + "/lm/en_US/" + lm);
  c.setString("-rawlogdir", sphinxDirectory.getAbsolutePath());
  ...

Only the constructor changed. This should allow for us to use other dictionaries and lm files fairly easily without having to muck with RecognizerTask.

The next step was to eliminate the unnecessary functionality of PocketSphinxDemo. I removed the performance calculations and edited the code. The author did something funny with making “that” references—which were final references tothis. I’m not sure the reasoning and would welcome explanation from the erudite among you. I changed the IDs of the widgets to more meaningful ones. I also triggered the asset-to-storage copy. This class was the result:

package edu.cmu.pocketsphinx.demo;

import java.io.File;
import org.twodee.android.utils.Log;
import org.twodee.android.utils.Utilities;
import android.app.Activity;
import android.os.Bundle;
import android.view.MotionEvent;
import android.view.View;
import android.view.View.OnTouchListener;
import android.widget.Button;
import android.widget.EditText;

public class PocketSphinxDemo extends Activity implements RecognitionListener {
  static {
    System.loadLibrary("pocketsphinx_jni");
  }

  /** Recognizer task, which runs in a worker thread. */
  RecognizerTask recognizer;

  /** Are we listening? */
  boolean isListening;

  /** Text box for results. */
  EditText editText;

  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    setContentView(R.layout.main);

    // Copy out assets directory to storage. TODO: condition this on files
    // missing.
    Utilities.copyAsset(this, "pocketsphinx");

    // Set up widgets.
    editText = (EditText) findViewById(R.id.resultsText);

    Button b = (Button) findViewById(R.id.listenButton);
    b.setOnTouchListener(new OnTouchListener() {
      @Override
      public boolean onTouch(View v,
                             MotionEvent event) {
        if (event.getAction() == MotionEvent.ACTION_DOWN) {
          isListening = true;
          recognizer.start();
        } else if (event.getAction() == MotionEvent.ACTION_UP) {
          isListening = false;
          recognizer.stop();
        }
        return false;
      }
    });

    isListening = false;
    recognizer = new RecognizerTask(new File(getExternalFilesDir(null), "pocketsphinx"), "digger_dict.dic", "digger_lm.lm");
    recognizer.setRecognitionListener(this);
    new Thread(recognizer).start();
  }

  public void onPartialResults(Bundle b) {
    final String hyp = b.getString("hyp");
    editText.post(new Runnable() {
      public void run() {
        editText.setText(hyp);
      }
    });
  }

  public void onResults(Bundle b) {
    final String hyp = b.getString("hyp");
    editText.post(new Runnable() {
      public void run() {
        editText.setText(hyp);
      }
    });
  }

  @Override
  public void onError(int err) {
    Log.d("Error: %d", err);
  }
}

It works, once. Subsequent listens fail.

Our solution is to mirror out the assets directory to storage the first time an app is run (or whenever that directory disappears). These two methods recursively copy the asset structure we want to mirror:

/**
 * Copy the asset at the specified path to this app's data directory. If the
 * asset is a directory, its contents are also copied.
 * 
 * @param path
 * Path to asset, relative to app's assets directory.
 */
private void copyAsset(String path) {
  AssetManager manager = getAssets();

  // If we have a directory, we make it and recurse. If a file, we copy its
  // contents.
  try {
    String[] contents = manager.list(path);

    // The documentation suggests that list throws an IOException, but doesn't
    // say under what conditions. It'd be nice if it did so when the path was
    // to a file. That doesn't appear to be the case. If the returned array is
    // null or has 0 length, we assume the path is to a file. This means empty
    // directories will get turned into files.
    if (contents == null || contents.length == 0)
      throw new IOException();

    // Make the directory.
    File dir = new File(getExternalFilesDir(null), path);
    dir.mkdirs();

    // Recurse on the contents.
    for (String entry : contents) {
      copyAsset(path + "/" + entry);
    }
  } catch (IOException e) {
    copyFileAsset(path);
  }
}

/**
 * Copy the asset file specified by path to app's data directory. Assumes
 * parent directories have already been created.
 * 
 * @param path
 * Path to asset, relative to app's assets directory.
 */
private void copyFileAsset(String path) {
  File file = new File(getExternalFilesDir(null), path);
  try {
    InputStream in = getAssets().open(path);
    OutputStream out = new FileOutputStream(file);
    byte[] buffer = new byte[1024];
    int read = in.read(buffer);
    while (read != -1) {
      out.write(buffer, 0, read);
      read = in.read(buffer);
    }
    out.close();
    in.close();
  } catch (IOException e) {
    Log.e(e);
  }
}