If Ikea made instructions for Selenium

How to fix bugs in Selenium for yourself

Daniel Wagner-Hall and the Selenium Committers

dawagner@gmail.com

@illicitonion

selenium-developers@googlegroups.com

irc.freenode.net #selenium

http://bit.ly/seconf-workshop

Daniel Wagner-Hall and the Selenium Committers

dawagner@gmail.com

@illicitonion

selenium-developers@googlegroups.com

irc.freenode.net #selenium

http://bit.ly/seconf-workshop

How to fix a bug

  • Test Case
  • Diagnosis
  • Treatment
  • Verification
  • Patch please!

Test Case

  • Static hosted content
  • Minimal - keep reducing!
  • Simple JUnit/rspec/py.test/NUnit/... test case of form:

test() {

WebDriver driver = new ChromeDriver();

driver.get("http://127.0.0.1:8888/foo");

driver.findElement(By.id("bar")).click();

assertEquals("Expected title", driver.getTitle());

}

  • REPLs are nice

We have quite a few!

  • ./go test_firefox
  • ./go test_opera method=testTheOnlyTestMethodToRun
  • ./go test_chrome onlyrun=ClassOfTestsToRun
  • common/src/web -> localhost:XXXXX
  • Note: ./go debug-server serves on http://localhost:2310/common

  • New test:
    • Add a new HTML file to common/src/web
    • Add a test method where it fits

Sources of Truth

  • Code
  • Wiki: https://code.google.com/p/selenium/w
  • People:
    • irc.freenode.net #selenium
    • selenium-developers@googlegroups.com

Architecture

  • One "driver" per browser
    • Page interaction
    • State
    • Formatting
  • One "binding" per language (thin RPC layer)
    • Mostly type-translation
    • Mostly stateless
  • Talk JSON over HTTP - RESTish
    • Except Safari (WebSockets), HtmlUnit, Opera (Java)
  • RemoteWebDriver server, Grid act as proxies

...

Extension

Scope

COM

Automation proxy

Extension

WebView Wrapper

WebView Wrapper

HTTP

WebSockets

...

Extension

Scope

COM

Automation proxy

Extension

WebView Wrapper

WebView Wrapper

HTTP

WebSockets

The Atoms

  • Cross-browser user-interaction abstraction
  • Project, pulled out from WebDriver and Selenium core
  • Lives in javascript/atoms
  • Some layers on top, e.g.:
    • javascript/webdriver-atoms
    • javascript/selenium-atoms
  • bot.* namespace
  • Tests!
    • ./go test_javascript
    • ./go debug-server
    • http://localhost:2310/javascript

Closure: A quick note

  • Compiles, optimises, minifies javascript
  • Large-ish browser-compat and java-ish abstraction library
  • goog.require => "Allow me to use"
  • goog.provide => "goog.require this file in order to use"
  • Type annotations in comments

Entry Points

  • The language API call actually being made
  • Which will lead to an RPC, probably HTTP, through some:
    • C#,Java: *CommandExecutor
    • Python: remote_connection.py
    • Ruby: bridge.rb

FirefoxDriver

  • Firefox extension, installed on-the-fly
  • Lives in javascript/firefox-driver
  • [A bit of a lie]: One big compiled blob of javascript in js/ [depends on the atoms]
  • Flow of command:
    • webdriverserver.js ->
    • dispatcher.js ->
    • nsCommandProcessor.js ->
    • {firefoxDriver,nsCommandProcessor, wrappedElement}.js
  • With a little help from wdsession.js, and a respond callback-object

FirefoxDriver Debugging Tips

  • fxdriver.Logger.dumpn -> Firefox error console
  • fxdriver.Logger.dump
  • FirefoxProfile.preference["webdriver.log.file"] will save logs to file
  • WebDriver in red means a command has been dispatched but hasn't yet sent result
  • Use the SynthesizedFirefoxDriver in Java

ChromeDriver

  • Lives in chromium codebase: https://src.chromium.org/svn/trunk/src/chrome/test/webdriver
  • Links against the whole of chromium - SLOW
  • Flow of command:
    • webdriver_server.cc ->
    • webdriver_dispatch.cc ->
    • commands/* ->
    • webdriver_session.cc ->
    • Chrome automation API

IEDriver

  • C++ driving IE through COM
  • ATL smart pointers for memory management
  • Lives in cpp/IEDriver
  • Also pulls in the atoms as Generated/atoms.h
  • Flow of command:
    • WebDriver.cpp ->
    • IEServer.cpp ->
    • IESession.cpp ->
    • IECommandExecutor.cpp ->
    • CommandHandlers/*
  • Also depends on cpp/webdriver-interactions for typing, mouse events

IEDriver Debugging Tips

  • Build IEDriverServer exe, run from build/cpp/{Win32,x64}/Debug/IEDriverServer.exe - connect to http://127.0.0.1:5555
  • printf
  • Hook VS debugger

HtmlUnitDriver

  • Wraps a WebClient pretty tightly
  • Lives in java/client/src/org/openqa/selenium/htmlunit
  • Basically two 1000 line java classes:
    • HtmlUnitDriver.java
    • HtmlUnitWebElement.java

Debugging tips:

  • printf
  • Java debugger

SafariDriver

  • Safari Extension - loaded manually
  • Promise-based asynchronous javascript
  • postMessage message passing between components
  • Lives in javascript/safari-driver
  • Safari extension - see Info.plist
  • Global Page:
    • extension/extension.html -
    • extension/extension.js#safaridriver.extension.init() ->
    • extension/server.js ->
    • extension/commands.js
  • Message passes to Content Script:
    • inject/inject.js ->
    • inject/commands.js via message.js

SafariDriver Debugging Tips

  • Uses closure's logger, which outputs to the error console
  • Plenty of examples in code

OperaDriver

  • Java, talks scope over protocol buffers to Opera itself
  • OperaDriver.java and follow java calls to scope RPC boundary
  • Scope specification at http://dragonfly.opera.com/app/scope-interface/

AndroidDriver

  • Android application containing WebView which we control
  • android project in android/ bootstraps java library in java/client/src/org/openqa/selenium/android/library
  • AndroidApkDriver, MainActivity wrap an AndroidWebDriver, which has some (slightly weird) pure-java calls, with a custom ViewClient and ChromeClient to listen for events, and poke a WebView

Notes:

  • UI-thread requirements
  • Object creation expensive
  • Synchronizing on "this" expensive

AndroidDriver Debugging

  • Use a real device. Please.
  • Gingerbread (2.3)+, newer is better

  • set DEVID to the device ID listed by $ANDROID_SDK/platform-tools/adb devices
  • Build: ./go clean //android:android-server
  • Install: $ANDROID_SDK/platform-tools/adb -s $DEVID -e install -r build/android/android-server.apk
  • Start: $ANDROID_SDK/platform-tools/adb -s $DEVID shell am start -a android.intent.action.MAIN -n org.openqa.selenium.android.app/.MainActivity
  • $ANDROID_SDK/platform-tools/adb -s $DEVID forward tcp:8080 tcp:8080
  • Connect to http://localhost:8080/wd/hub

iWebDriver

  • Wraps WebView, just like AndroidDriver
  • Fairly monolithic:
    • RootViewController ->
    • MainViewController, WebViewController, HTTPServerController
  • Connect to http://localhost:3001/wd/hub

RemoteWebDriver

  • Proxy middleman servlet
  • Lives in java/server/src/org/openqa/selenium/remote/server
  • Command flow:
    • DriverServlet.java ->
    • [some magical DI in ResultConfig] ->
    • handler/**/* ->
    • Delegate to reference to underlying WebDriver, with element ID creation in KnownElements

Let's Get Going!

Jari - Ruby

Dave B - Python

Jim - IE, C#

Dave H - IDE

Jason

Kristian - Grid, Threading

Jason - Safari, JSAPI, iWebDriver

Kevin - Grid

Eran - Interactions, Native events

Francois - Grid

Simon

Santi

Andreas - Opera

Daniel

Samit - IDE

Jari - Ruby

Dave B - Python

Jim - IE, C#

Dave H - IDE

Jason

Kristian - Grid, Threading

Jason - Safari, JSAPI, iWebDriver

Kevin - Grid

Eran - Interactions, Native events

Francois - Grid

Simon

Santi

Andreas - Opera

Daniel

Samit - IDE