The Browser Client

Ellen Spertus (spertus@google.com)

This document describes the portion of App Inventor that runs in the user’s web browser, namely the client and shared portions of the appengine sub-project.  This does not cover the Blocks Editor code, which currently runs in a separate JNLP task but is being moved into the browser.

Background

Google Web Toolkit

Google App Engine

Relationship with other parts of App Inventor

Static dependencies

Runtime interactions

GUI elements

Panels

Design Tab

MotdBox

PaletteBox

ViewerBox

SourceStructureBox

AssetListBox

PropertiesBox

Projects Tab

Debugging Tab

Start-up Behavior

Background

Google Web Toolkit

Google Web Toolkit (GWT) allows programmers to write client-server applications in Java without worrying about the details of remote procedure calls (RPCs) except for providing explicit callbacks for RPCs (one for a successful call, one for failure).  GWT compiles the client code into JavaScript, which runs within a web browser, and the RPCs run as Java code on the server, with communication done via HTTP.  The reader is advised to learn about GWT before delving into the portion of App Inventor that runs in, or responds to requests from, the user’s browser.

Google App Engine

Google App Engine (GAE) is a cloud-computing platform that enables programs written in Java (or Python) to run and maintain data on Google servers.  The original internal version of App Inventor was built directly on proprietary Google infrastructure but, for the open source release, was rewritten to use GAE, using the third-party Objectify datastore API.

GWT and GAE play well together, where a GWT server can run on a GAE server.  This is how App Inventor works, as shown in Figure 1.

Figure 1: Block diagram of the App Inventor client and server.  
Both are created with GWT, which converts the front-end code into JavaScript, which is run with the GWT client library in the user’s browser.  The back-end runs on the GWT server library as a Google App Engine service, using the third-party Objectify API for data storage.

Relationship with other parts of App Inventor

Static dependencies

The App Engine client YaClientApp depends on:

Runtime interactions

Figure 2 shows dynamic interactions among the App Inventor sub-projects.  When the browser client starts, a variety of information (discussed in great detail in section Start-up Behavior below), is passed from the App Engine server to the client.  This includes the server build id and the message of the day (motd).  Information about a project’s components, assets, and user settings flows in both directions.

When the Blocks Editor is launched from the client via a Java WebStart file (not shown), the Blocks Editor requests and receives the current project’s component information and blocks code.  After any changes, the Blocks Editor sends the blocks code back to the server.  When components are added, deleted, or renamed in the client, the client sends updated component information to the server and notifies the Blocks Editor that a change to the components was made.  The Blocks Editor then requests and receives updated component information from the server.  This multi-step update is performed because only small amounts of information can be passed through the HTTP connection between the client and the Blocks Editor.  The full list of messages sent by the client to the Blocks Editor can be found in CodeblocksConnection.

If the user establishes a connection from the Blocks Editor to the device (or emulator), the Blocks Editor requests and receives AiPhoneApp (the APK that provides the “repl” interpreter) and the current project’s assets, which it passes through to the device.

If the user requests in the client that the application be built, the client sends a request to the server, which packages up the component information, blocks code, and keystore into a zip file, which it sends to the build server.  After attempting to build the application, the build server sends status/error messages and the APK to the App Engine server, which saves a copy of the APK and sends it and the messages to the client.  If the initial request was to build the application and download it to the phone (as opposed to producing a bar code or downloading it to the computer), the client sends a message to the Blocks Editor, which then requests and receives the APK from the App Engine server.

Figure 2: Runtime interactions among App Inventor sub-projects.  Requests for data are not shown.
The notifications sent by the client to the Blocks Editor alert it ask the server for updated properties, components, projects, or APKs.
More detail on the App Engine (browser) client and server can be seen in Figure 1.

GUI elements

Panels

The major panels are shown in Figure 3.  Not shown is the RootPanel, which is provided by GWT and fills the application window.  The remaining panels are created in Ode.initializeUi().  The first to be created is mainPanel, which fills the RootPanel.  Its children are of type:

Figure 3: Panels created in Ode.initializeUi(), outlined and labeled in magenta.  
The outermost panel, mainPanel, is of type
DockPanel and contains three panels: a TopPanel, a DeckPanel, and a StatusPanel.  The TopPanel and StatusPanel are present whether the view is of the Projects tab, Design tab, or Debugging tab.  The remainder of the picture is a DeckPanel (artificially reduced in size above).  It has three children; the one that gets displayed depends on the selected tab.

Design Tab

The Design tab, shown in Figure 4, is where the user adds components to a project, specifies their properties and positions, and manages media files (assets).

Figure 4: Design tab view.  The main VerticalPanel is shown in magenta, boxes in blue, and the DesignToolbar in dark purple.  
There are 5 boxes visible on this screen: MotdBox, PaletteBox, ViewerBox, SourceStructureBox, PropertiesBox, and AssetListBox.  The StatusPanel at the bottom has been omitted.

All of the box classes highlighted in Figure 4 implement the singleton pattern and directly extend the abstract class com.google.appinventor.client.widgets.boxes.Box, a container widget built on top of com.google.gwt.user.client.ui.FlowPanel that automatically handles scrolling for embedded widgets and can be resized, minimized, and restored.  Every box has a caption, which can be found in OdeMessages.  The boxes are added to the GUI in the private method Ode.initializeUi().

MotdBox

The MotdBox displays the message of the day (MOTD).  During startup, the method Ode.setupMotd() invokes the GetMotdService.getCheckInterval() RPC to find out how often to check for updates to the MOTD. Once a response is retrieved, the client constructs a MotdFetcher, which causes its fetchMotd() method to invoked, invoking the MotdService.getMotd() request.  When the MOTD is retrieved, MotdUi.setMotd() is called to display the MOTD.

PaletteBox

The PaletteBox contains a single element, a YoungAndroidPalettePanel, a Composite containing a StackPanel, which “stacks its children vertically, displaying only one at a time, with a header for each child which the user can click to display”.  In Figure 2, the “Basic” category is being displayed.

ViewerBox

The ViewerBox contains a ProjectEditor (specifically, a YaProjectEditor). The EditorManager maintains multiple ProjectEditor instances, one for each project opened in the current browser session.  In turn, each ProjectEditor has one FileEditor for every screen in the project.  This is illustrated in Figure 5.

Figure 5 also shows that each YaProjectEditor has a reference to a YaFormEditor, which manages much of the communication between GWT (drags and drops) and the boxes and panels.  For example, if a new component is dragged onto the YaFormEditor (YaFormEditor.onComponentAdded()), SourceStructureBox and PropertiesBox both get notified.

To provide “mock” representations of real components, such as the Screen, Button, and Sound in Figure 4, there is a hierarchy of mock components rooted in the class MockComponent.  Non-visible components, such as Sound, are represented by MockNonVisibleComponent, which has minimal functionality.  MockButton is one of the more complex mock components, since its appearance depends on the value of its properties, such as Text, Image, and BackgroundColor.

Figure 5: Static UML class diagram for ProjectEditor.  There is a single EditorManager, which maintains a collection of ProjectEditor instances, one for each project that has been opened in this browser session.  Each ProjectEditor maintains a collection of FileEditor instances, one for each screen in the project.  ProjectEditor also maintains the GUI elements shown in Figure 6.

Figure 6: A ViewerBox, with GUI components labeled (left) and unlabeled (right).  
A ViewerBox contains a TabBar within a
ScrollBar and a DeckPanel, which contains a SimpleNonVisibleComponentsPanel and a SimpleVisibleComponentsPanel, which contains a single VerticalPanel.  The VerticalPanel contains a CheckBox and a MockForm, which contains a MockForm.PhoneBar, a MockForm.TitleBar, and an AbsolutePanel in a ScrollPanel.  Not shown on the left (because they are not present in all projects) are the MockButton within MockForm and the MockNonVisibleComponent in the SimpleNonVisibleComponentsPanel.

SourceStructureBox

The sole item in a SourceStructureBox is a SourceStructureExplorer, which, as shown in Figure 6, displays a hierarchical view of the project’s components.  It is implemented as a Tree where each TreeItem contains a SourceStructureExplorerItem.

AssetListBox

The AssetListBox lists the assets (media files) uploaded by the programmer.  It contains an AssetList, which extends Composite and has a private member assetList, which is a Tree where each TreeItem contains a ProjectNode -- specifically, a YoungAndroidAssetNode (Figure 7).  The context menu commands available when a node has been selected, specified in CommandRegistry, are DeleteFileCommand and DownloadFileCommand.

Figure 7: Direct and indirect subclasses of ProjectNode.  
Not shown is the interface HasAssetFolder<YoungAndroidAssetsFolder>, implemented by YoungAndroidProjectNode.

PropertiesBox

The PropertiesBox, like the other boxes, is created in Ode.initializeUi(), although it is not populated until a component is selected.  Specifically, when YaFormEditor is constructed, it creates a new PropertiesPanel and saves a private reference to it named designProperties.  The contents of this panel are updated when any of the following occur:

Projects Tab

The Projects tab, shown in Figure 8, is where a user creates, deletes, opens, uploads, or downloads projects and keystores.  The ProjectToolbar, at top, contains four instances of ToolbarItem, each of which has a name (e.g., “DownloadAll”), a caption (e.g., “Download All Projects”, and a Command (e.g., ProjectToolbar.NewAction).  The ProjectListBox is populated by ProjectList, which contains a table with “Name” and “Date Created” editors that can be used for sorting, and rows consisting of a CheckBox, a project name, and a creation date.

Figure 8: Project tab view.  It consists of a VerticalPanel containing a ProjectToolbar (magenta) and a HorizontalPanel containing a ProjectListBox.  ProjectToolbar extends Toolbar, which extends Composite, and consists of a HorizontalPanel containing two HorizontalPanels, named leftButtons and RightButtons (currently unused).  ProjectListBox is populated with a ProjectList, which extends Composite and consists of a VerticalPanel containing a Grid, whose “Name” and “Date Created” headers are HorizontalPanels and each support sorting in either direction.  Each row consists of a CheckBox, a Label containing the project’s name, and a Label containing the project’s creation Date.

Debugging Tab

The Debugging tab (Figure 8) is offered only if AppInventorFeatures.hasDebuggingView() returns true.  It consists of a WorkAreaPanel that contains two boxes.  The first, MessagesOutputBox, displays messages passed to MessagesOutput.addMessages() about the status of requests to build the application and to download it to the phone.  OdeLogBox displays the output of messages logged through one of the following static methods:

Figure 9: Debugging tab view.  The upper box, MessagesOutputBox displays messages from the BuildServer.  
The lower box, OdeLogBox displays logging messages generated within the client code.

Figure 10: Static UML class diagram for Debugging tab components.
UI elements are shown in blue.  Not shown (because they are not App Inventor code) are the superclass of ColumnLayout.Column,
which is
AbstractIndexedDropController, and the superclass of Widget, which is UIObject.  
Both MessagesOutputBox and OdeLogBox are subclasses of Box.

Start-up Behavior

Figure 11: Start-up information flow and control flow.  
RPCs are indicated with horizontal arrows between the client (left) and server (right), where queries and responses have the same color and initial number.  Colored dashed arrows indicate local method and constructor calls that occur after the associated response is received.  Black dotted lines indicate local method calls due to listeners.  For example, Ode.onModuleLoad() makes one RPC (2Q).  When a response is received, it makes three local calls: UserSettings.loadSettings(), ProjectManager(), and Ode.initializeUi(), which calls Ode.setupMotd().

Figure 11 shows the information and control flow when a user begins an App Inventor session (assuming there are no RPC failures or other exceptional occurrences).  Note that this part of the document is even more subject to change than the rest.

  1. User navigates in browser to the URL of an App Inventor server, which sends an HTTP GET request [1Q] to the App Engine server (produced by the appengine project).
  2. The App Engine server returns the client JavaScript code [1R].
  3. The client browser begins running the code in Ode.onModuleLoad(), which invokes the UserInfoService.getUserInformation() RPC to get basic information about the user [2Q].
  4. The server returns a User data object [2R], consisting of the signed-in user’s unique identifier, email address, and whether the user has accepted the terms of service (TOS). (The following discussion assumes that the user has already accepted the TOS.  If not, the RPC fails, and the client redirects the user to a page where they can view and accept the TOS.)  The callback is within Ode.onModuleLoad(), which calls the following methods/constructors:
  1. UserSettings.loadSettings(), which invokes the UserInfoService.loadUserSettings() RPC to find out the id of the last project opened by the user [3Q].  (Because all requests are asynchronous, the client advances to the next step before receiving a response.)
  2. ProjectManager(), which invokes the ProjectService.getProjectInfos() RPC to get basic information about all of the user’s projects [4Q].
  3. Ode.initializeUi(), which sets up the user interface for the in-browser application.  It then calls Ode.setupMotd(), which invokes the GetMotdService.getCheckInterval() RPC to find out how often to check for updates to the message of the day (MOTD) [5Q].

For the sake of this discussion, we assume that queries 3Q, 4Q, and 5Q are answered in the order in which they were issued, but responses could come in any order.

  1. RPC UserInfoService.loadUserSettings() returns a JSON-encoded String with the user’s settings, including the id of the last opened project [3R].  This launches a series of local method calls.  The last one, Ode.openProject(), locally registers a listener to invoke Ode.openYoungAndroidProjectInDesigner() when the project is added to the client’s list of known projects, which will occur as a result of query 4Q in step 4b above.
  2. RPC ProjectService.getProjectInfos() returns a List<UserProject> with summary information about each project, such as its id, name, and most recent modification date [4R].  For each of these, a call is made to ProjectManager.addProject(), which invokes any registered listeners, namely the one from step 5 waiting for the user’s most recent project to be added.
  3. Ode.openYoungAndroidProjectDesigner() is called, which itself calls Project.loadProjectNodes() and invokes the RPC ProjectServices.getProject() to get complete information about the last-opened project [6Q].
  4. The server responds to GetMotdService.getCheckInterval() with the number of seconds the client should wait between checking for changes to the MOTD [5R].  The client constructs a MotdFetcher(), which causes its fetchMotd() method to be invoked, invoking the MotdService.getMotd() to request the MOTD [7Q].  (Subsequent requests for the MOTD are tacked onto other RPC requests; see MotdFetcher.onStart().)
  5. The server responds to query 6Q with complete information about the user’s last opened project [6R], upon which Project.fireProjectLoaded() calls Ode.openYoungAndroidProjectDesigner() which opens the Designer view to this project.
  6. Finally, the server returns the MOTD [7R], which the MotdUi.setMotd() method writes to the user interface.

Subsequent control flow and RPCs depend on the user’s behavior.