Tuesday, July 20, 2010

Using Objectify to access the datastore

There are several options to choose from when deciding how you want to access the App Engine datastore. Two options are Java Data Objects (JDO) and Java Persistence API (JPA); both are from Sun and are in the Java SDK. From searching around the web, the consensus seems to be that both are too heavyweight for most App Engine projects. They are, however, both standards, and JDO is datastore agnostic, which would make moving your application off of the App Engine platform somewhat easier. However, you still are stuck with the limitations of the GAE datastore, so it's not quite so easy bringing JDO code into your GAE project.

Open source alternatives for Google App Engine


Two GAE specific open source projects are Objectify and Twig. Out of the two, Objectify seems simpler and more aligned with what's actually happening in the datastore. As I learn GAE, I feel it's somewhat imperative to not have the details of the datastore hidden from me. It's also modeled after the GAE Python library, which seems pretty clean. And, last but not least, your data classes can be passed through GWT RPC without modification. Perhaps in the future I'll come to a different conclusion if I get tired of having to write too much in the way of nuts and bolts that Twig handles automatically. Twig also supports parallel queries and merging OR queries, both of which could be pretty nice.

Hearing about the different design philosophies straight from the horses' mouths gave me some good information I needed to make this decision, too.

The Objectify concepts documentation is a good starting point for understanding both Objectify, and the GAE datastore.

Adding Objectify to VolInfoMan


Adding Objectify to the project leads us to the first time that we need a servlet running on App Engine. The servlet will simply provide a front end to Objectify and the datastore by responding to GWT RPC calls.

The code for this state of the project is available as tag "add-object-step1" on github.

He's not any kind of program, Sark. He's a User.


The first step is to define the object we want to pass back and forth between our GWT code and the GAE servlet. To continue the login page we've been working on, we'll need a User so we can authenticate the session. Since the User will be shared between GWT and GAE, we'll put it in the shared package.

(I'm leaving out the package and import statements to conserve space)

com/lisedex/volinfoman/shared/User.java
public class User implements Serializable {
        @Id
        private Long id;
        @Indexed
        private String username;
        @Indexed
        private long status;
        @Unindexed
        private String firstName;
        @Unindexed
        private String lastName;
        @Unindexed
        private String email;
        @Unindexed
        private String password;

Objects that will go through RPC need be Serializable. Later on, I'll probably add the @Cached annotation to the class so that it will automatically be cached by GAE's memcache.

You can only run queries on the GAE datastore against fields that are indexed. More specifically, you need an index for every query you intend to run, so by default all class fields are indexed to make it easier to query on any specific field. However, according to the Objectify best practices documentation, each property that is indexed requires a separate write to the index, which won't add to latency (writes are done in parallel), but will add to the CPU time used by the application. So instead, I explicitly declare which fields are to be indexed, and which are not. The same could be achieved by declaring the whole class @Indexed and only specifying the @Unindexed, or vice versa, but I like to explicitly declare each for easier reading. The field to be used to create the Key for the object is annotated with @Id.

I put in a couple of getter/setters, and a toString() method for easy logging. It will need to be fleshed out more later.

GAE, we need to have a serious talk. - Love, GWT


Next, we need to define the interface our client uses to access the servlet. I put GWT code for handling data into com.lisedex.volinfoman.client.data, so I define a UserService interface in that package.

com/lisedex/volinfoman/client/data/UserService.java
@RemoteServiceRelativePath("user")
public interface UserService extends RemoteService {
        User getUser(Long id);
        User getUser(String username);

        void putUser(User user);
}

The important part is that we declare what URL we'll be using to access the RPC servlet using the @RemoteServiceRelativePath annotation. The interface itself extends RemoteService, which all GWT RPC client interfaces should extend, and then we add the RPC functions we want to support. I want to support grabbing a User by name or id number, or putting a User object back into the datastore.

If you're using Eclipse, it will complain about the lack of a UserServiceAsync interface, and will build it for you automatically if you let it. It's basically the same as above, but doesn't need to extend RemoteService or declare a @RemoteServiceRelativePath. You can grab the complete code from github, as shown above.

In the server package, we need a servlet class that implements the UserService interface we just defined.

com/lisedex/volinfoman/server/UserServiceImpl.java
public class UserServiceImpl extends RemoteServiceServlet implements
                UserService {

    private DAO dao = new DAO();

    public User getUser(String username) {
            return dao.getUser(username);
    }
}

The real code has stub implementations for getUser(Long id) and putUser(User user) since we're not using them for now. We want to abstract away how we'll be accessing the datastore in case we need to change it, so we put actual interaction in a Data Access Object called...DAO.

com/lisedex/volinfoman/server/DAO.java
public class DAO extends DAOBase {
        static {
                ObjectifyService.register(User.class);
        }

        private Objectify ofy;

        public DAO() {
                ofy = ObjectifyService.begin();
        }

We don't have to extend the DAOBase class, but it gives us a couple of functions that we can use without coding, such as a lazily instantiated Objectify object. Of course, I didn't know that at the time I wrote the code, so I created my own. Oops.

You need to register the classes you'll be using with Objectify, and calling ObjectifyService.begin() returns an Objectify object we'll use to interact with the datastore.

public User getUser(String username) {
                User fetched = ofy.query(User.class)
                    .filter("username", username).get();
                return fetched;
        }

        public User getOrCreateUser(String username) {
                User fetched = ofy.query(User.class)
                    .filter("username", username).get();
                if (fetched == null) {
                        fetched = new User(null, username, 
                            User.STATUS_INVALID, null, null, null, 
                            null);
                        ofy.put(fetched);
                }
                return fetched;
        }

        public void putUser(User user) {
                ofy.put(user);
        }
}

To get a User by username, we need to execute a query on the datastore for all User objects, and filter the results by the username field. If no such object is in the datastore, the get() method will return a null.

Writing the object is easier, since we already know the Key for the object, as it's encoded in the class (User.class, and Long id). Just pass the object to Objectify's put() method. If the id field in the object is of type Long, and its value when passed to put() is null, Objectify will automatically generate an id for the object. If the id is of type long or String, the developer is responsible for that themselves.

The getOrCreateUser() function should be modified to use the get() and put() methods in DAO. Another oops. And I need to add more data checking, such as if the User sent to put() is null.

Make the Login button do something


Now we need to wire this stuff into the user interface, so that when we hit the Login button, we actually go out to the datastore and try to retrieve the appropriate User.

com/lisedex/volinfoman/client/DefaultHomepage.java
class MyHandler implements ClickHandler {

    @Override
    public void onClick(ClickEvent event) {
        sendStatus.setText("Sending..." + username.getText() +
                           "/" + password.getText());
        sendButton.setEnabled(false);
        userService.getUser(username.getText(),
            new AsyncCallback<User>() {
                  public void onFailure(Throwable caught) {
                      sendStatus.setText(sendStatus.getText()
                                 + "   FAILED");
                      sendButton.setEnabled(true);
                  }

                  @Override
                  public void onSuccess(User result) {
                      if (result == null) {
                             sendStatus.setText(sendStatus.getText() 
                                 + "    NO SUCH USER");
                             sendButton.setEnabled(true);
                             return;
                      }
                      sendStatus.setText(sendStatus.getText()
                          + "    SUCCESS: " + result.toString());
                      sendButton.setEnabled(true);
                  }
              });
    }
}

sendButton.addClickHandler(new MyHandler());

We obviously have not implemented the MVP model, as all of this stuff is in our view. In the ClickHandler inner class, we call the UserService.getUser() asynchronous method with an AsyncCallback inner class that will handle what happens when the RPC call returns.

If the RPC call fails, onFailure gets called with an exception. This doesn't happen if the username doesn't exist in the User indexes; it only happens with the client is not able to access the servlet for some reason. onSuccess is called with a User object when the RPC call works. The result is null if no such User was found, otherwise it's populated with all stored fields for the object.

Next up: Guice, security


Guice is the basis for Gin, except Guice provides dependency injection for Java generally, where Gin is for GWT. DAO's are a perfect place to apply dependency injection, since we may want to use the real datastore back end, or a dummy used for testing. We'll cover that in the next installment.

Also, there's no security or authorization needed to call any of the RPC functions. Anyone that wants can read or write any User object to my datastore, even if they're not using my client application. We'll, uh, get to that sometime.

No comments:

Post a Comment