ABFIJKLMNOQZ
1
CategorySubCategoryJBehave FitnesseGreenPepperCucumberGUIdancerJubulaPyWinautoRobot FrameworkSelenium Web Testing ToolTelerik Testing Framework
2
Note: The row start with high-level attributes and get more detailed lower down.
3
Documentation Write user stories and acceptance criteria (scenarios) in plain-text files in any language.Use Automated Functional Testing as an "Executable Specification" mechanism that optimize requirement communication between customers/users/field experts and the developers team. All this in order to build the right software, the one needed.Write business scenarios in structured any natural language (English, French, Welsh,..)Same goals as Jubula, but with added extras for professional use (code coverage, TestStyle, Mylyn integration, reporting)Functional acceptance testing (black-box) through
the GUI to test programs as the end user will see them.

Removal of programming effort from acceptance testing.

Use in agile processes (tests created in parallel with
development).		Robot Framework is a generic keyword-driven test automation framework for acceptance level testing and acceptance test-driven development (ATDD)Selenium tests web apps on any OS in any browser with any language.Free framework for UI automation of HTML web pages, Silverlight applications (in- or out-of-browser), and Windows Presentation Foundation desktop applications. Supports cross-browser playback in IE, Safari, Firefox, and Chrome.
4
ImplementationNative LanguageJava or any other language that you can access via Java API, e.g. groovy, jruby, jython, rhino.JavaJava, .NET, RubyRuby, Java, Groovy, Clojure, Jython, JRuby, Scala, Rhino JavaScript, Ioke, JavaScript, Lua, C++, C#, F#
JavaJavaPythonAll of themC# or VB.NET
5
Implementation.Net AccessibleRequires Java/.NET interopYesYesYes - SpecFlow (http://www.specflow.org) targets the .NET framework, Silverlight, Windows Phone and MonoNoNoIronPythonYesYes
7
InterfaceAPI/WebUI/DesktopGUIYes, Selenium-based Steps already exist.APIAPIaSee JubulaJava Swing, SWT/RCP, WebUI, GEFPluggableAPI/WebUIAPI, Web UI, Desktop, any .NET or Win32 similar APIs
16
Languagevariables as arguments of keywordsVariables are matched with text in story. Default implementation is regex-basedNoNoScenario OutlineYesYesYesTests are code in a programming language.Yes, via standard programming constructs.
17
Languagesupports variablesVariables are matched with text in story. Default implementation is regex-basedNoNoScenario OutlineYesYesYesTests are code in a programming language.Yes, via standard programming constructs.
18
LanguageData-drivenYesYesYesYesYesTests are code.Yes, via standard programming constructs.
20
LanguageProgramming Language is used to write testsTest is plain text, Execution framework is JavaNoNo - plain text (Gherkin)NoNoYesYes.
28
MiscMixed Auto/Manual InstallN/AN/AYesYesYes.
29
MiscOpenSourceYesYesAll runners are opensourceYesNo, professional version of JubulaYesYesYesYesFree, but not open source.
30
MiscCommercialNoPay for Server and technical supportIt's freeYesNoNoNoYes. Paid support, also complete test automation toolset on top of framework.
34
ProcessAcceptance Test Driven developmentYesYesYesYesYesYesYesTests are code in a programming language.Yes, via standard programming constructs and working with something like Cucumber, Fitness, MSpec, etc.
38
ReportingSee screen capturesPossible - example existsN/ANoPossible - see watir exampleYesYesDepends on Driver ("Tool")YesCan capture screenshots if devs and testers decide to code that up.
39
ReportingAggregate test resultsYes - customizableYesYesYesYesYesYes. Tests are code in a programming language.Depends on test runner
45
RoleReportingConsole, HTML, XML, text, "stats", customYesHTML in WIKI (Confluence, XWiki), JIRA or on disk (HTML)Console, HTML, XML, PDF, customSee Jubula. Additionally, long term BIRT trends can be generated from results.In client itself (as summary table and full report tree).
Also as HTML and XML reports.
YesOut of scope for an automation toolDepends on test runner.
46
RoleTest EditorAny text editorYesConfluence, XWikiAny text editor, RubyMine, NetBeans. Bundles exist for textmate, Vim, etc)See Jubula. Additionally, Mylyn tasks can be used for context-based working.
There is also a Test Style module for best practices for test structure etc. 		In "Test Case Editor": tests can be added (referenced),
removed and altered (e.g. order, names, components, data)		YesSelenium IDE. For RC:Vim, Emacs, etc.Depends on test runner.
47
RoleTest RunnerAnt, Maven, JUnit, customYesFrom a Wiki (XWiki, Confluence), from JIRA or from the IDE (Eclipse & Visual Studio plugin). Runner versions for Java, .Net and Ruby). Using continuous integration with Maven, Nant, MS Build.Command line, Rake, IDESee JubulaYes: Tests can be started from a command line,
which can be called by e.g. Ant or CI tools		YesYesAny environment that can invoke the testing framework's internal runner, or via NUnit, MsTest, Fitness, etc.
60
Test EditorRecord TestsWith Selenium, for Selenium-based StepsNoNoNoYesYesNoYes (Selenium IDE)N/A. This is the role of your code editor.
61
Test EditorInteractive executionWhen run in IDE via JUnit/TestNGNoNoYes, use the ask function in a step definitionYesYesYes, Selenium IDE. For RC, use an IDE like EclipseN/A. This is the role of your code editor.
67
Test RunnerGUI/BrowserYes - whatever you want to add to Steps class - Selenium/WebDriver classes existBrowserYesYes - Watir, WebDriver, Celerity, Capybara, Webrat compatibleYesYesYesYes, Selenium IDEDepends on test runner.
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108