Add Gesture Search to your Android apps

By Yang Li, Research Scientist

Gesture Search from Google Labs now has an API. You can use the API to easily integrate Gesture Search into your Android apps, so your users can gesture to write text and search for application-specific data. For example, a mobile ordering application for a restaurant might have a long list of menu items; with Gesture Search, users can draw letters to narrow their search.


Another way to use Gesture Search is to enable users to select options using gestures that correspond to specific app functions, like a touch screen version of keyboard shortcuts, rather than forcing hierarchical menu navigation.

In this post, I’ll demonstrate how we can embed Gesture Search (1.4.0 or later) into an Android app that enables a user to find information about a specific country. To use Gesture Search, we first need to create a content provider named CountryProvider, according to the format required by Android Search framework. This content provider consists of 238 country names.

Then, in GestureSearchAPIDemo, the main activity of the app, we invoke Gesture Search when a user selects a menu item. (Gesture Search can be invoked in other ways depending on specific applications.) To do this, we create an Intent with the action "com.google.android.apps.gesturesearch.SEARCH" and the URI of the content provider. If the data is protected (for example, see AndroidManifest.xml), we also need to grant read permission for the content URI to Gesture Search. We then call startActivityForResult to invoke Gesture Search.

public boolean onCreateOptionsMenu(Menu menu) {
    super.onCreateOptionsMenu(menu);
    menu.add(0, GESTURE_SEARCH_ID, 0, R.string.menu_gesture_search)
        .setShortcut('0', 'g').setIcon(android.R.drawable.ic_menu_search);
    return true;
  }

  @Override
  public boolean onOptionsItemSelected(MenuItem item) {
    switch (item.getItemId()) {
      case GESTURE_SEARCH_ID:
        try {
          Intent intent = new Intent();
          intent.setAction("com.google.android.apps.gesturesearch.SEARCH");
          intent.setData(SuggestionProvider.CONTENT_URI);
          intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION);
          intent.putExtra(SHOW_MODE, SHOW_ALL);
          intent.putExtra(THEME, THEME_LIGHT);
          startActivityForResult(intent, GESTURE_SEARCH_ID);
        } catch (ActivityNotFoundException e) {
          Log.e("GestureSearchExample", "Gesture Search is not installed");
        }
        break;
    }
    return super.onOptionsItemSelected(item);
  }

In the code snippet above, we also specify that we want to show all of the country names when Gesture Search is brought up by intent.putExtra(SHOW_MODE, SHOW_ALL). The parameter name and its possible values are defined as follows:

/** 
   * Optionally, specify what should be shown when launching Gesture Search.
   * If this is not specified, SHOW_HISTORY will be used as a default value.
   */
  private static String SHOW_MODE = "show";
  /** Possible values for invoking mode */
  // Show the visited items
  private static final int SHOW_HISTORY = 0;
  // Show nothing (a blank screen)
  private static final int SHOW_NONE = 1;
  // Show all of date items
  private static final int SHOW_ALL = 2;

  /**
   * The theme of Gesture Search can be light or dark. 
   * By default, Gesture Search will use a dark theme.
   */
  private static final String THEME = "theme";
  private static final int THEME_LIGHT = 0;
  private static final int THEME_DARK = 1;

  /** Keys for results returned by Gesture Search */
  private static final String SELECTED_ITEM_ID = "selected_item_id";
  private static final String SELECTED_ITEM_NAME = "selected_item_name";

As you can see in the code, when Gesture Search appears, we can show a recently selected country name, or nothing. Gesture Search then appears with a list of all the country names. The user can draw gestures directly on top of the list and a target item will pop up at the top. When a user taps a country name, Gesture Search exits and returns the result to the calling app. The following method is invoked for processing the user selection result, reading the Id and the name of the chosen data item.

@Override
  protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (resultCode == Activity.RESULT_OK) {
      switch (requestCode) {
        case GESTURE_SEARCH_ID:
          long selectedItemId = data.getLongExtra(SELECTED_ITEM_ID, -1);
          String selectedItemName = data.getStringExtra(SELECTED_ITEM_NAME);
          // Print out the Id and name of the item that is selected 
          // by the user in Gesture Search
          Log.d("GestureSearchExample", selectedItemId + ": " + selectedItemName);
          break;
      }   
    }
  }

To use the Gesture Search API, you must be sure Gesture Search is installed. To test this condition, catch ActivityNotFoundException as shown in the above code snippet and display a MessageBox asking the user to install Gesture Search.

You can download the sample code at http://code.google.com/p/gesture-search-api-demo.

Yang Li builds interactive systems to make information easily accessible anywhere anytime. He likes watching movies and spending quality time with his family.

Posted by Scott Knaster, Editor