Google Cloud Platform: Mobile Endpoints

• Google Cloud Platform: Getting Started with Google App Engine (Part 1)
• Google Cloud Platform: Deploying with Google App Engine (Part 2)
• Google Cloud Platform: User Management (Part 3)
• Google Cloud Platform: Mobile Endpoints (Part 4)
• Google Cloud Platform - Google Cloud SQL (Part 5)
• Google Cloud Platform - Google Cloud Datastore (Part 6)
• Google Cloud Platform - Google Cloud Storage (Part 7)

Part 4: Mobile Endpoints

In the late 90’s, when the Web was just getting started, the emphasis was on building applications—user interface was in HTML, databases were accessed from the web server, and so on. Supporting "multiple clients" meant supporting IE, Firefox, Opera, and (later) Chrome. "Client-side execution" meant Javascript running in the browser, and "client-side data storage" meant cookies. For close to a decade, this was the model almost everybody followed.

Then, mobile technology reached its critical mass, and smartphones, followed by tablets and now other mobile devices (Google Glass, "smart watches", and more to come) began to dominate the developer landscape. Supporting "multiple clients" now means (at a minimum) Android, iOS, and whatever desktop OS’es are running on laptops, tablets, or even desktops. "Client-side" execution means native applications running on those devices, and "client-side data storage" can mean anything from SQLite running on the device to NoSQL options to… well, sometimes even a full-fledged RDBMS. And all this, while still supporting browser-based front-ends (which, by the way, may be running on the aforementioned devices).

The need to support a variety of different front-ends has led to a change in architecture—no longer is it always browser-to-webserver-to-database; instead, developers need (and want) to build REST endpoints that provide a single point-of-access for data and logic. Now, instead of the HTTP server slinging back HTML, it sends XML or (increasingly) JSON data, without concern for which client is receiving the data, since both of those are platform-neutral formats. In fact, to use the parlance of the early 2000’s, the web server has been transformed from a tool of the client layer, into the application server as a whole.

The act of building servlets that transform Java objects into XML or JSON can be something of a pain, however. Granted, numerous libraries and toolkits exist in open-source packages all across the Internet to make this easier, but Google figured if they could enhance the server-side infrastructure so that developers wouldn’t have to constantly grab (and update, and so on) those modules, it would make everybody’s life easier. Remember: nothing prevents a developer from doing a servlet-sending-JSON directly within Google App Engine, but give the Cloud Endpoints a spin before you make any decisions.

Cloud Endpoints

In point of fact, a large part of the value of the Google Cloud Endpoints lies in its ability to generate client-side libraries that "hide" the details of the HTTP, preparing the data for transport (marshaling it into JSON and back again), the actual HTTP URL, and so on. Granted, this may not seem like a huge task—after all, just grab one of those open-source JSON libraries and maybe the Apache HttpClient library and you’re good to go—but Google’s Cloud Endpoints will not only generate client-side libraries for Java (assuming an Android front-end, basically), but also JavaScript (for rich client-side Web apps) and iOS (for everybody’s favorite fruity mobile devices), but do so based on the current source of the server-side classes. If that’s a tad confusing, don’t stress: it’ll make more sense once we see the process in action.

Let’s start by starting over.

Begin at the beginning

Our demo, all along so far, has been the creation of a "greeting" service that says howdy; when we were building Web pages, it was generated HTML, but since now we’re thinking more along the "API" route, we want to generate a JSON response. Rather than try adapting the servlet-based code from earlier, let’s just start a new project. Easiest way to do that is to grab the "new_project_template" from within the Google App Engine SDK (it’s hidden away under "demos"), and copy it wholesale over to a working directory for some hacking. Let’s call this project "helloapi", and since it’s going to generate a WAR file like the demo from Part 2 did, let’s keep it all local for the moment to keep things a little simpler. The project template has a sample servlet and an index.html file that we won’t need, so kill them off to keep things simple (or not, if you want to use them as a quick verification test when deploying either locally or to the cloud—either way, they’re not something we use further).

One other change is needed to the project template to make it all work: the Ant script included in the root of the project directory needs to know where the Google App Engine SDK is located:

<!-- When you copy this template to your working directories, change the
    value of appengine.sdk accordingly.  You may also want to use Ant's
    property file feature, allowing each developer to have their own
    local property file with the path to the SDK from that workspace. -->
<property name="appengine.sdk" location="C:/Prg/appengine-java-sdk-1.8.7"/>

Additionally, one change that’s officially optional but one that I aesthetically prefer is to change the name of the output directory in which the WAR contents are generated; the project template calls it "www", but I’m old-school enough that I think WARs should be assembled in a "war" directory:

<!-- Change if you like e.g. "war" better than "www" for the output -->
<property name="war.dir" location="war"/>

Call me old-fashioned.

Code

The first thing is to create the endpoint itself: a standard Java class that exposes one or more public methods that take zero or more parameters and hand an object or a collection of objects back. In other words, a pretty plain ol’ Java object:

package com.tedneward.appenginedemo;

import com.google.api.server.spi.config.Api;
import javax.inject.Named;
 
class Message
{
    public String message;
   
    public Message(String m) { message = m; }
   
    public String getMessage() { return message; }
    public void setMessage(String value) { message = value; }
}
 
public class Greetings
{
    public Message greet(String target)
    {
        return new Message(
            "Hello, " + target + ", from Google Cloud Endpoints!");
    }
}

It may seem strange that "greet" is returning an object with a public field, instead of just a String; although a simple String might work for this example, in general, a REST API is going to want to hand back a full data transfer object (DTO) containing more than one field of data; in some cases, those structures will be entirely non-trivial. Note, however, that these DTOs shouldn’t be actual "domain objects", but flattened versions of them that are designed for easy consumption on the client (which, remember, won’t always be Java).

The endpoint isn’t done—there’s a few things that need to be added before the Cloud Endpoints tooling will recognize it as an endpoint—but this is the end of the "coding" part of the exercise, for the most part. And, in fact, that’s the point: the Cloud Endpoints functionality allows the Java developer to focus on the coding, not the mechanics of HTTP or JSON.

Annotations

The code needs a couple more things to fit into the Cloud Endpoint view of the world. First, it needs an "Api" annotation (from the com.google.api.server.spi.config package) to indicate the "name" of this endpoint (which will be used as part of the HTTP URL path), and a "version" (for similar reasons):

package com.tedneward.appenginedemo;

import com.google.api.server.spi.config.Api;
import javax.inject.Named;
 
class Message
{
    public String message;
   
    public Message(String m) { message = m; }
   
    public String getMessage() { return message; }
    public void setMessage(String value) { message = value; }
}
 
@Api(name = "helloworld", version = "v1")
public class Greetings
{
    public Message greet(String target)
    {
        return new Message(
            "Hello, " + target + ", from Google Cloud Endpoints!");
    }
}

Notice that the "greet" method expects a parameter, and that’s hardly unusual for REST APIs—in fact, it’s more the common case. REST APIs, however, are based on HTTP, which means that they’re either positioned as part of the URL, or else are submitted as JSON named parameters. Again, the goal of Cloud Endpoints is to leave those kinds of distinctions behind, so here we use the Java-standard "Named" annotation (from the javax.inject package) to indicate that the parameter being passed is coming in from the API, and is expected to have the same name as the method parameter itself:

package com.tedneward.appenginedemo;

import com.google.api.server.spi.config.Api;
import javax.inject.Named;
 
class Message
{
    public String message;
   
    public Message(String m) { message = m; }
  
    public String getMessage() { return message; }
    public void setMessage(String value) { message = value; }
}
 
@Api(name = "helloworld", version = "v1")
public class Greetings
{
    public Message greet(@Named("target") String target)
    {
        return new Message(
            "Hello, " + target + ", from Google Cloud Endpoints!");
    }
}

The "Named" annotation can expose any name, but to keep things sane, it’s usually convenient to keep the exposed name the same as the method parameter.

Both of these packages come from outside the list of libraries included by default as the default project’s Ant compile-classpath, so a quick edit to the Ant script to the "compile" target to include the JARs in <appenginesdk>/lib/opt/user/appengine-endpoints/v1 will be necessary:

<fileset dir="${appengine.sdk}/lib/opt/user/appengine-endpoints/v1">
    <include name="*.jar"/>
</fileset>

Make sure this appears in both the <javac> task and the <copy> task (which copies the JARs into the WEB-INF/lib directory in the assembled target). Currently, there’s only one JAR file in there, but the App Engine SDK does a pretty good job of separating out the JARs by functionality/feature, so this way the Ant script is future-proofed in the event that Google adds a few more JARs in there, or splits the one JAR up into constituent parts.

Administrivia

A couple of administrative elements remain that the project needs in order to be a full-fledged Google Endpoint: a description (metadata) about the endpoint API needs to be present in the WEB-INF directory, and in order to do that, we need to register a servlet in the WAR’s web.xml file that knows about our endpoint and can reflect and describe it.

The web.xml fix is pretty straightforward:

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app
   xmlns="http://java.sun.com/xml/ns/javaee"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
   version="2.5">
  <display-name>HelloAPI</display-name>
 
  <servlet>
    <servlet-name>SystemServiceServlet</servlet-name>
    <servlet-class>
      com.google.api.server.spi.SystemServiceServlet
    </servlet-class>
    <init-param>
      <param-name>services</param-name>
      <param-value>com.tedneward.appenginedemo.Greetings</param-value>
    </init-param>
  </servlet>
 
  <servlet-mapping>
    <servlet-name>SystemServiceServlet</servlet-name>
    <url-pattern>/_ah/spi/*</url-pattern>
  </servlet-mapping>
 
  <!-- the rest as-is -->
</web-app>

This servlet, when invoked, will take care of generating the necessary metadata to describe the endpoint; it’s the server half of the next step, which is to run a command-line utility, "endpoints" (a shell script for UNIXers, a batch file for Windowsers), asking it to generate an "API" file that must be stored in the "WEB-INF" directory:

$ endpoints gen-api-config -o src\WEB-INF com.tedneward.appenginedemo.Greetings
Nov 17, 2013 5:51:21 PM com.google.apphosting.utils.config.AppEngineWebXmlReader
 readAppEngineWebXml
INFO: Successfully processed ./war\WEB-INF/appengine-web.xml
API configuration written to src\WEB-INF/helloworld-v1.api

Without this API file in the WEB-INF directory, the Endpoints host will throw a 500 error, claiming a 404 error as part of the stack trace. And it will need to be regenerated each time the public API of the Greetings class or its dependents (such as Message) change, so really, this step should be folded in as part of the Ant script; given that this isn’t an article on Ant, I’ll leave that as an exercise to the reader.

Note that running the "endpoints" script reveals an interesting bug: gen-api-config isn’t listed as one of its acceptable "verbs", but there’s no other way to generate one of these API files.

Testing

At this point, fire up the App Engine development web server, either by using "ant runserver", or fire it up directly using "dev_appserver". Testing this can take one of several forms: one, hit the URL with cURL, but that implies that we know what the URL for the endpoint is, which isn’t obvious until you see it in action, or two, use the built-in API explorer that comes as a part of the Google Endpoint, by firing up the browser and pointing it to "http://localhost:8080/_ah/api/explorer", and using the UI to navigate to the API endpoint in question ("helloworld API v1" > "helloworld.greetings.greet"). Fill in the red-labeled fields in the form (those are the required parameters), click the big blue "Execute" button, and not only will we see the greeting response, but it shows the HTTP request that generated it. In this case, the URL is http://localhost:8888/_ah/api/helloworld/v1/greet/fred, which is hardly intuitive until we break it down: "/_ah/api" is the prefix for all Endpoint APIs, "helloworld" and "v1" both come from the @Api annotation, "greet" is the method name tied to the class (Greeting) that holds the @Api annotation, and "fred" is our parameter.

By the way, when looking at the API explorer view, see that "OAuth" switch in the upper right corner? That’s because by judicious use of annotations (look at the Api class and ApiAuth class in the Javadocs), we can flag certain methods as requiring OAuth authentication for use, while leaving others as public. In fact, the Api annotation has about two dozen different fields on it, allowing for some really deep customization of the endpoint itself, including configuration of API quotas (limits applied to unregistered users of the API).

Clienting

Again, all of this could’ve been done using servlets and manual transformation into JSON and back again, but the real win comes when writing the front-end to the REST API we just built; the "endpoints" command can generate a complete library that hides the details of invoking the API in question. So, in this particular case, to generate an Android-compatible library, it’s

$ endpoints get-discovery-doc com.tedneward.appenginedemo.Greetings

… to generate a "discovery" doc that is then fed to endpoints a second time, like so:

$ endpoints get-client-lib ./helloworld-v1-rest.discovery

… and assuming all is successful, it will generate a .ZIP file containing the client library code. This file is going to be somewhat bigger than expected—it’s intended to be a "batteries-included" client-side library, meaning it will not only contain a JAR file with the compiled client code in it, but also JAR files for all of that client’s dependencies, along with license files and documentation. In short, it’s a fully-redistributable bundle, with everything that any client would ever need.

Summary

Building client-friendly REST APIs is not going to "just happen" from using the Google Cloud Endpoints toolkit—there’s still the task of thinking from a REST-based perspective, identifying the resources and how they’re manipulated via the core HTTP verbs—but the nice thing about Cloud Endpoints is that developers can focus on the REST parts, and not all the surrounding infrastructure necessary to make REST happen in a Java application. And, should the client that needs to be built be something other than Android, iOS or JavaScript-powered (Windows8, I’m looking at you), the API is still accessible to anyone with an HTTP connection and a basic understanding of the JSON format.

Sometimes, though, in addition to handing back a greeting, an API wants to know how many times a given client called it, including the timestamp when they called, and that calls for data storage, something we’ll get into next time. In the meantime, though, have fun being greeted, and happy coding!