-
Notifications
You must be signed in to change notification settings - Fork 0
Suite Syntax
vdobler edited this page Jun 5, 2011
·
2 revisions
The following is the reference-suite.wt and describes all features.
#
# Reference Suite for Webtest
# ===============================
#
# Webtest test suites are simple text files in UTF-8 (without BOM).
# Line endings may be \n or \r\n and you are stubid _and_ ugly if you use
# the later. The line length should not exeed 4000 characters.
#
# Lines "starting" with a # are comments and are ignored. The # need
# not be the very first character on a line: # is considered 'start of
# comment' iff it is the first non-whitespace charater on a line.
#
# Individual test cases are introduced like this (test case name surrounded
# by lines of - signs):
# ------------------------------------
# The Name of the Test Case
# ------------------------------------
# See below: Global is such a testcase. Unfortunately Global is as
# very special testcase. So please skip to the "Ping" test case.
#
# Notes on test names: Test names should be uniq in one suite and they
# should not contain commas (",") in the name. Rational: This allows
# to select individual tests from a suite during a webtest run with
# the -tests option.
#
-------------------------------
Global
-------------------------------
GET http://unused.for/global
RESPONSE
# Normly we expect to get 200. Overwrite in test if you need to
# test for e.g. 404
Status-Code == 200
SETTINGS
# Any request taking longer than 45 seconds is considered an error.
Max-Time 45000
CONST
# The global BaseUrl of our test server.
BaseUrl http://localhost:8080
#
# General Structure of Test Cases
# -------------------------------
#
# The most basic and simple test case: See if a server answers.
#
# After the test case header (-------\n<Name>\n-------) several
# sections describe various aspects of the test. A Section is intruduced
# by an all caps section name starting at the first character of a line.
# Individual setting in a setting are indented by (at least) a tab '\t'.
#
# The first section is special: It names the request method and URL and
# must be the first section, cannot be omitted an has no sub-elements.
# All other section are optional and may occur in any order. The Ping
# test below contains just the RESPONSE section.
---------------------------------
Ping
---------------------------------
# Method may be "GET" or "POST". The URL must be a valid, full qualified
# URL. https works like expected.
GET http://host.to.ping/path.html
# The response section: In this section the various header fields of the
# response can be checked.
RESPONSE
# Check that the server answered with 200 status code.
Status-Code == 200
# Thats all for the ping test case
#
# Checking Response Header Fields
# -------------------------------
#
# More on response fields checking: Recieved header fields can be accessed
# by their name (no trailing colon), e.g. "Content-Type".
# There are two special fields which can be checked allways, even if the
# server didn't include them in the response header: "Status-Code" and
# "Final-Url". Status-Code is the numerical status code and Final-Url is
# the URL reached after doing all the redirects requested by the server.
#
# There are several ways to test the recieved value.
---------------------------------
Ping Enhanced
---------------------------------
GET http://host.to.ping/path.html
RESPONSE
# Operator == is for real equality
Status-Code == 200
# Operator ~= tests for "contains"
Content-Type ~= text/html
# Operator /= tests for a regular expression matching the field value
# and is considered for expert use.
Strange-Field /= (cat?|^dog?).+$
# Operator _= tests for the field value starting with the given prefix
Other-Header _= StartPrefix
# Operator =_ tests for the field value ending with the given suffix
Something =_ EndSuffix
# For field which are numeric you may use <, <=, ==, >= or > with the
# usual meaning.
Content-Length > 500
# For fields whose value is a RFC1123 date you may use also use
# <, <=, > and >=. (Only RFC1123 dates work, but others should not
# be used anyway...)
Expires >= Fri, 20 May 2011 12:59:19 GMT
# To negate a condition: Prefix the whole condition with a ! charcter.
# Note: There is no != operator. Use !field == val.
!App-Field ~= Illegal
# To disalow the mere existence of a header field:
!Illegal-Header
# Generally there is no need to quote field names or values to test
# against: field names do not contain whitespace or special characters
# and values are just the rest after the operator with leading and
# trailung whitespace trimmed.
# If you do need these leading or trailing whitespaces: Enclose the
# value with " marks:
Field-Name == " spaces at begin and end are important for the test "
#
# Setting Request Header fields
# -----------------------------
#
# You may specify any request header field in the HEADER section by just
# naming them and their value. Note: There is a special syntax for sending
# cookies: Refere to section Cookies.
---------------------------------
Ping With Header
---------------------------------
GET http://host.to.ping/path.html
# The header section: add special request header fields here.
HEADER
# add the Accept-Language fields with given value. Note: No colon (:)
Accept-Language de,fr,en
# Quotes could be used around the value to include leading or trailing
# spaces, but request header fields normaly do not contain spaces at
# all.
#
# Basic Authorization
# --------------------
#
# To use basic authorization you can provide the Authorization header
# yourself, or you can use the special fake Basic-Authorization header.
---------------------------------
Ping With Basic Authorization
---------------------------------
GET http://host.to.ping/path.html
HEADER
# This is a pure convenience feature.
# username and password are in cleartext and will be properly
# base64 encoded. No "Basic-Authorization" header will be sent,
# instead a correct Authorization header will be sent:
# Authorization: Basic=<base64 encoded user credentials>
Basic-Authorization username:password
#
# Testing the Response Body
# -------------------------
# There are three ways to test the content of the recieved body:
# Simple tests are specified in the BODY section, HTML tags can be checked
# in the TAG section.
---------------------------------
Ping Content
---------------------------------
GET http://host.to.ping/path.html
# All simple test are placed in the body section
BODY
# Txt is the whole text of the body
Txt == Whole text of body
# Attention: Only UTF-8 encoded bodys work well.
# Bin ist the whole text of the body as hexadecimal string
Bin == 0daf23bcad873f94
# Note: Syntax may change in the future
# The same operators (without the numerical ones) like in the response
# section can be used with with Txt and Bin: ~=, _=, =_ and /=
# Test if boby contains "Hello World!"
Txt ~= Hello World
# Use something like this to check if a binary file starts with the
# appropriate magic key (png magic key below)
Bin _= 89504e470d0a1a0a
# HTML/XML tag/element checkings are placed in section TAG
# Syntax for the tags are like checktag. See documentation in tag.go.
# The syntax is a bit like
# tagSpec := ['!'] [ numOp number ] { simpleTag | tagStructure }
# numOp := { '<', '<=', '==', '>=', '>' }
# number := <any number >= 0, e.g. 4 or 17>
# simpleTag := tagName [class...] [attribute...] [ { '==' | '=D='} content]
# tagName := <the lower case tag name: h2, div, iframe, ...>
# class := [ '!' ] 'class='content
# attribute := [ '!' ] attrName'='content
# attrName := <the lowercase attribute name, e.g. href, title, ...>
# content := { '/' regexp '/' | pattern }
# regexp := <a valid regular expression>
# pattern := <a text pattern with * and ? as the usual wildcards>
# tagStruct := '[' '\n' moreIndnt { simpleTag | tagStructure } '\n' ']'
# moreIndnt := <more indentation by tabs/spc than previous/parent tagSpec.>
#
TAG
# Check if any h2 tag with a class of 'home' and text content of
# 'Quality' is present
h2 class=home == Quality
# Fail if a h5 tag with content 'WRONG' is present
! h5 == WRONG
# Whitespaces in text content is normalized: tabs and newlines are
# replaced by spaces, multiple spaces are collapsed to one and
# leading/trailing spaces are trimmed. I.e. the text content of
# <p> Hello John Doe!
# Greetings!
# </p>
# is considered to be "Hello John Doe! Greetings!" and would be
# matched by
p == Hello John Doe! Greetings!
# but not by p == Hello John Doe! Greetings!
# Count occurences of this div tag with CSS class teaser:
# Must be exactly 3
=3 div class=teaser
# span tags may not be present 2 times (0, 1, 3, .. or 17) is okay
!=2 span == xyz
# Fail if there are more than 4 a-tags linking to /somewhere.html.
<5 a href=/somewhere.html
# The rest of the numerical operators are <=, >= and > and work like
# expected.
# Negations are discuraged (but allowed): !<=, !>=, !< and !>
# Tag structures (nested tags) are introduced by '[' and ended by ']'
# Each on a own line.
[
div class=A
div class=B
ul
]
# Negation '!' and counting operators may be placed before the [.
# Test if this div with span element occurs at least 5 times.
>4 [
div class=X
span == Test
]
# Note: The following structure would match any teaser (the div) which
# contains as direct childs a h3- and a p-tag, regardless if there are
# other tags present: So
# <div class="teaser>
# <h1>Super</h1>
# <h3>Freibier</h3>
# <span>Heute!</span>
# <p>Die ganze Woche Freibier für alle</p>
# </div>
# Would match the following structure.
[
div class=teaser
h3 == Freibier
p == Die ganze Woche*
]
# Checking for "deep" content: Consider the following html
# <h2> Hello<span>nice</span>World</h2>
# The text content of the h2 is considered to be "Hello World"
# (note the trimming of spaces and the addition of a space
# between the two text nodes).
# To match the whole text content including nested tags use
# the "=D=" deep operator
h2 =D= Hello nice World
#
# Sending Parameters
# ------------------
#
# Arbitary parameters can be specified in the PARAM section.
# For GET requests they are appended to the given URL. POST requests
# are sent as "multipart/form-data" iff a file is uploaded and
# "application/x-www-form-urlencoded" if not.
# See below for forcing multipart posts.
---------------------------------
Post a Comment
---------------------------------
POST http://my.blog/comment.html
PARAM
# Parameter name and value are given like this.
date 2010-03-04
# As a parameter may have seveal values you need to quote values
# with spaces as a space is the delimiter for the different values.
name "Grigori Perelman"
text "A Proof of the Poincare Conjecture"
# Sending multiple values of a parameter (e.g. checkbox)
categorie genius nerd
# If one of the multiple values contains a space: Surround _this_
# one with quotes
categorie proof "hilbert problem" theorem
# To send a file use the following syntax:
file @file:relative/path/to/document.pdf
# Currently there is a bug: You may not send filenames with
# special characters (e.g. spaces)
#
# Forcing multipart/form-data
# ---------------------------
#
# To force the post to use "multipart/form-data" even if no file
# is uploaded: Use POST:mp as method.
---------------------------------
Force Multipart Upload
---------------------------------
POST:mp http://my.blog/comment.html
PARAM
name anonymous
comment Cool!
#
# Variable Subtitutions
# ---------------------
#
# There are three types of variable substitutions, all can be used like shell
# variables: const variables (just sounds strange), sequence variables and
# random variables.
# A variable can be asigned a value and used as part of the URL, part of
# the header, response, body, tag or parameter values. The usage is allways
# the same for all three types: Occurences of ${<varname>} are replaced by
# the value of the varibale <varname>.
# But there are three ways to set a value for a variable in the three
# sections:
# - CONST variables just take a single value, theire use is obvious.
# - SEQ (sequence) variables and
# - RAND (random) values,
# The only reasonable use for sequence and random variables is in a repeated
# test: SEQ and RAND values take values of a given list of possible values.
# SEQ cycles through the list in the given order, whereas RAND picks one
# value by random for each round of the test.
#
# Notes:
# - Variable names consist of characters only (no numbers no _).
# - The following variable names are reserved for future use:
# "GLOBALID", "RANDOM", all variables starting with "ENV" and
# "NOW" (see below)
# - Pay attention if variables are substituted in tag content as this might
# generate a regexp: If x takes value "xyz/" and the tag spec is e.g.
# "p == /abc*${x}" it will result in "/abc*xyz/ which is considered a
# regexp.
#
---------------------------------
Variables
---------------------------------
# Usage of a variable: ${BaseUrl} is replace by const value set bolow.
# resulting in URL beeing http://my.blog/entries/show
GET ${BaseUrl}/show
PARAM
month ${Month}
year ${Year}
user user-${Name}
TAG
# Variable substitution is done in the content part of tags only:
# All other elements do not allow variables.
h2 == Hello ${Name}!
CONST
# Set value of BaseUrl to http://my.blog/entries. Note: No = sign!
BaseUrl http://my.blog/entries
SEQ
# Month will be 1 on first run of test, 2 on second, and so on.
# Will restart beeing 1 on 13th run of test.
Month 1 2 3 4 5 6 7 8 9 10 11 12
RAND
# Year is one of the given four selected randomly on each test run.
Year 2004 2005 2006 2007
# Values with spaces like 'Emil Tom' must be enclosed in quotes as
# usual
Name Anna "Emil Tom" Vicktoria
SETTING
# Repeat the test 7 times.
Repeat 7
#
# Special Veriables
# -----------------
#
# The following variables are provided by the system and cannot be
# redefined: "NOW" (currently the only one).
---------------------------------
Special Variables
---------------------------------
GET http://some.url
RESPONSE
# NOW is the current time formated as RFC1123 (taht is
# "Mon, 02 Jan 2006 15:04:05 MST"
Date == ${NOW}
# Now can be increased/decreased by adding/subtracting timespans.
# Formating remains RFC1123
Expires > ${NOW + 3days}
# If you need a different time format: add your own fmt definition
# on gos time format after a '|' charcter. Time will be in UTC!
Last-Modified >= ${NOW - 5 hours + 10 minutes | Mon Jan _2 15:04:05 2006}
# Possible modifiers are "second", "minute", "hour", "day", "week",
# "month" and "year" (all lower case plural accepted).
# Output is in UTC time format (to prevent bug in Go)
#
# Repeating Tests
# ---------------
#
# There are two different ways to "repeat" a test: Setting the the setting
# "Repeat" or "Tries" to a number greater 0.
# - "Repeating" a test n times means executing the the test n times and
# reporting a success only if all n individual tests succeed. This is
# usefull for iterating over sequence or random variables. See above
# for examples.
# - "Trying" means trying at most n times. Pass imediately if one run
# succeeds and fail if all n rounds fail. This is usefull to wait for
# some background job to complete and check this regularely.
# Note: Combining repetition and trying is possible, but the result is
# currently undefined (aka buggy): The test status is solely determined
# by the last repetition.
---------------------------------
Wait for Background Job
---------------------------------
GET http://some.host/job/123/detail.html
BODY
Txt ~= Job 123 finished.
SETTING
# Wait two seconds after each test
Sleep 2000
# Try at most 60 times: Fail if not finished after approx 2 minutes.
Tries 60
#
# Settings
# --------
#
# Various setting can be applied to each and every test in the SETTING
# section. Currently the following are implemented.
---------------------------------
Settings
---------------------------------
GET http://host.to.ping/path.html
SETTING
# Number of repetitions of the test. Set to 0 to "disable" this test.
Repeat 12
# Number of tries this test is executed at most. The test passes if
# one try succeeds (the rest of the possible tries are skipped) and
# fails if all tries fail.
Tries 5
# Setting both Tries and Repeat to values > 1 is (currently) undefined.
# Time in ms to sleep after test
Sleep 250
# Fail if answer is not recievd in less than 300 ms.
Max-Time 300
# Keep (store in Global) cookies set by the server answer. See
# below. Use 0 to turn storage off.
Keep-Cookies 1
# Dump (see below in Debugging)
Dump 0
# Abort test suite and fail immediately if this tests fails.
# Usefull to skip test which cannot be tested because some setup task
# failed.
Abort 1
#
# Cookies
# -------
#
# Cookies can be sent along with the request and recieved cookies
# (Set-Cookie) can be checked. Both have their own section.
-------------------
Cookies
--------------------
GET http://some.url
SEND-COOKIE
# The easier cookie stuff: Send cookie with name SESSIONID and value
# 123ABCxyz along with request
SESSIONID 123ABCxyz
SET-COOKIE
# The complicated stuff: Test wether server requests to set a cookie.
# make sure the server wants to set cookie with name "just-present"
# or make sure that no cookie "illegal" was set. (Values do not matter)
just-present
!illegal
# Two ways to check the value of the cookie, the first is just an
# abrevation for the later, the general form
name == full-value
name:Value ~= contained
# Check other fields of the cookie. Please note: As the dot '.' is not
# explicitely forbiden in cookie names, but ':' is, the colon is the
# field delimiter
name:Path _= /some/path
name:Secure == true
name:Domain == www.mydomain.org
name:HttpOnly == false
name:MaxAge == 0
name:Expires ~= Nov 2013
name:Expires > Mon, 02 Jan 2006 15:04:05 MST
# If the server sends a cookie which is allready expired or the max age
# is lower 0, this cookie is deleted and also deleted in Global if
# Keep-Cookies is true
# Note: Cookies must be explicitely deleted, mere expiering will
# _not_ remove them.
SETTING
# Recieved sookies can be stored in the SEND-COOKIE section of the
# Global test. (Usefull for login/session cookies).
Keep-Cookies 1
# Note: Currently the server cannot delete a cookie he set in a
# previous test (neither by Expires nor MaxAge) if Keep-Cookies
# was set.
#
# Debuging
# --------
#
# Setting Dump to 1 will dump the whole request/response talk to a
# .dump-file. The filename is constructed from the test name.
# Setting Dump to 3 will save the response body to a file.
-------------------
Debuging
--------------------
GET http://some.url
SETTING
# Turn dumping on with 1 or 2. Will dump to file "Debuging.dump"
# 1 will create a new file wheras 2 will append to an existing one.
# 1 and 2 will dump the whole wiretalk of request and response while
# 3 will just dump the recieved response body (and create a new file
# each time)
Dump 1
#
# Global
# ------
#
# If the very first test in a test suite is named "Global" than this test
# will not be run, but serve as a template for all subsequent tests:
# Settings, Variables, Header fields and Response checks are inherited
# from global to each test. The test may overwrite them. Body and Tag
# checks cannot be overwritten (as the do not contain some uniq id to
# identify them): Body and Tag conditions/checks from global are just
# added to each test.
#
# Global is also used to keep cookies, e.g. login cookies which should
# be present in subsequent tests.
#
# See Global-"Test" above for an example