Packages

Emysql - Erlang MySQL driver

Current section

Files

Jump to
emysql test test.cfg
Raw

test/test.cfg

NOT USED
{suites, test, [environment_SUITE, emysql_SUITE]}.
%% %% %% http://www.erlang.org/doc/apps/common_test/run_test_chapter.html#id215610
%% %%
%% %% 6.8 Using test specifications
%% %%
%% %% The most flexible way to specify what to test, is to use a so called test
%% %% specification. A test specification is a sequence of Erlang terms. The terms may
%% %% be declared in a text file or passed to the test server at runtime as a list
%% %% (see run_testspec/1 in the manual page for ct). There are two general types of
%% %% terms: configuration terms and test specification terms.
%% %%
%% %% With configuration terms it is possible to e.g. label the test run (similar to
%% %% ct_run -label), evaluate arbitrary expressions before starting a test, import
%% %% configuration data (similar to ct_run -config/-userconfig), specify HTML log
%% %% directories (similar to ct_run -logdir), give aliases to test nodes and test
%% %% directories (to make a specification easier to read and maintain), enable code
%% %% coverage analysis (see the Code Coverage Analysis chapter) and specify
%% %% event_handler plugins (see the Event Handling chapter). There is also a term for
%% %% specifying include directories that should be passed on to the compiler when
%% %% automatic compilation is performed (similar to ct_run -include, see above).
%% %%
%% %% With test specification terms it is possible to state exactly which tests should
%% %% run and in which order. A test term specifies either one or more suites, one or
%% %% more test case groups, or one or more test cases in a group or suite.
%% %%
%% %% An arbitrary number of test terms may be declared in sequence. Common Test will
%% %% by default compile the terms into one or more tests to be performed in one
%% %% resulting test run. Note that a term that specifies a set of test cases will
%% %% "swallow" one that only specifies a subset of these cases. E.g. the result of
%% %% merging one term that specifies that all cases in suite S should be executed,
%% %% with another term specifying only test case X and Y in S, is a test of all cases
%% %% in S. However, if a term specifying test case X and Y in S is merged with a term
%% %% specifying case Z in S, the result is a test of X, Y and Z in S. To disable this
%% %% behaviour, it is possible in test specification to set the merge_tests term to
%% %% false.
%% %%
%% %% A test term can also specify one or more test suites, groups, or test cases to
%% %% be skipped. Skipped suites, groups and cases are not executed and show up in the
%% %% HTML test log files as SKIPPED.
%% %%
%% %% When a test case group is specified, the resulting test executes the
%% %% init_per_group function, followed by all test cases and sub groups (including
%% %% their configuration functions), and finally the end_per_group function. Also if
%% %% particular test cases in a group are specified, init_per_group and end_per_group
%% %% for the group in question are called. If a group which is defined (in
%% %% Suite:group/0) to be a sub group of another group, is specified (or particular
%% %% test cases of a sub group are), Common Test will call the configuration
%% %% functions for the top level groups as well as for the sub group in question
%% %% (making it possible to pass configuration data all the way from init_per_suite
%% %% down to the test cases in the sub group).
%% %%
%% %% Below is the test specification syntax. Test specifications can be used to run
%% %% tests both in a single test host environment and in a distributed Common Test
%% %% environment (Large Scale Testing). The node parameters in the init term are only
%% %% relevant in the latter (see the Large Scale Testing chapter for information).
%% %% For details on the event_handler term, see the Event Handling chapter.
%% %%
%% %% Config terms:
%% %%
%% %% {node, NodeAlias, Node}.
%% %% {init, InitOptions}.
%% %% {init, [NodeAlias], InitOptions}.
%% %% {label, Label}.
%% %% {label, NodeRefs, Label}.
%% %% {multiply_timetraps, N}.
%% %% {multiply_timetraps, NodeRefs, N}.
%% %% {scale_timetraps, Bool}.
%% %% {scale_timetraps, NodeRefs, Bool}.
%% %%
%% %% {cover, CoverSpecFile}.
%% %% {cover, NodeRefs, CoverSpecFile}.
%% %%
%% %% {include, IncludeDirs}.
%% %% {include, NodeRefs, IncludeDirs}.
%% %% {config, ConfigFiles}.
%% %% {config, NodeRefs, ConfigFiles}.
%% %% {userconfig, {CallbackModule, ConfigStrings}}.
%% %% {userconfig, NodeRefs, {CallbackModule, ConfigStrings}}.
%% %%
%% %% {alias, DirAlias, Dir}.
%% %% {merge_tests, Bool}.
%% %%
%% %% {logdir, LogDir}.
%% %% {logdir, NodeRefs, LogDir}.
%% %%
%% %% {event_handler, EventHandlers}.
%% %% {event_handler, NodeRefs, EventHandlers}.
%% %% {event_handler, EventHandlers, InitArgs}.
%% %% {event_handler, NodeRefs, EventHandlers, InitArgs}.
%% %% {ct_hooks, CTHModules}.
%% %% {ct_hooks, NodeRefs, CTHModules}.
%% %%
%% %%
%% %% Test terms:
%% %%
%% %% {suites, DirRef, Suites}.
%% %% {suites, NodeRefs, DirRef, Suites}.
%% %%
%% %% {groups, DirRef, Suite, Groups}.
%% %% {groups, NodeRefsDirRef, Suite, Groups}.
%% %% {groups, DirRef, Suite, Group, {cases,Cases}}.
%% %% {groups, NodeRefsDirRef, Suite, Group, {cases,Cases}}.
%% %% {cases, DirRef, Suite, Cases}.
%% %% {cases, NodeRefs, DirRef, Suite, Cases}.
%% %% {skip_suites, DirRef, Suites, Comment}.
%% %% {skip_suites, NodeRefs, DirRef, Suites, Comment}.
%% %%
%% %% {skip_cases, DirRef, Suite, Cases, Comment}.
%% %% {skip_cases, NodeRefs, DirRef, Suite, Cases, Comment}.
%% %%
%% %%
%% %% Types:
%% %%
%% %% NodeAlias = atom()
%% %% InitOptions = term()
%% %% Node = node()
%% %% NodeRef = NodeAlias | Node | master
%% %% NodeRefs = all_nodes | [NodeRef] | NodeRef
%% %% N = integer()
%% %% Bool = true | false
%% %% CoverSpecFile = string()
%% %% IncludeDirs = string() | [string()]
%% %% ConfigFiles = string() | [string()]
%% %% DirAlias = atom()
%% %% Dir = string()
%% %% LogDir = string()
%% %% EventHandlers = atom() | [atom()]
%% %% InitArgs = [term()]
%% %% CTHModules = [CTHModule | {CTHModule, CTHInitArgs} | {CTHModule,
%% %% CTHInitArgs, CTHPriority}]
%% %% CTHModule = atom()
%% %% CTHInitArgs = term()
%% %% DirRef = DirAlias | Dir
%% %% Suites = atom() | [atom()] | all
%% %% Suite = atom()
%% %% Groups = atom() | [atom()] | all
%% %% Group = atom()
%% %% Cases = atom() | [atom()] | all
%% %% Comment = string() | ""
%% %%
%% %%
%% %% Example:
%% %%
%% %% {logdir, "/home/test/logs"}.
%% %%
%% %% {config, "/home/test/t1/cfg/config.cfg"}.
%% %% {config, "/home/test/t2/cfg/config.cfg"}.
%% %% {config, "/home/test/t3/cfg/config.cfg"}.
%% %%
%% %% {alias, t1, "/home/test/t1"}.
%% %% {alias, t2, "/home/test/t2"}.
%% %% {alias, t3, "/home/test/t3"}.
%% %%
%% %% {suites, t1, all}.
%% %% {skip_suites, t1, [t1B_SUITE,t1D_SUITE], "Not implemented"}.
%% %% {skip_cases, t1, t1A_SUITE, [test3,test4], "Irrelevant"}.
%% %% {skip_cases, t1, t1C_SUITE, [test1], "Ignore"}.
%% %%
%% %% {suites, t2, [t2B_SUITE,t2C_SUITE]}.
%% %% {cases, t2, t2A_SUITE, [test4,test1,test7]}.
%% %%
%% %% {skip_suites, t3, all, "Not implemented"}.
%% %%
%% %%
%% %% The example specifies the following:
%% %%
%% %% * The specified logdir directory will be used for storing the HTML log files
%% %% (in subdirectories tagged with node name, date and time).
%% %% * The variables in the specified test system config files will be imported
%% %% for the test.
%% %% * Aliases are given for three test system directories. The suites in this
%% %% example are stored in "/home/test/tX/test".
%% %% * The first test to run includes all suites for system t1. Excluded from the
%% %% test are however the t1B and t1D suites. Also test cases test3 and test4 in t1A
%% %% as well as the test1 case in t1C are excluded from the test.
%% %% * Secondly, the test for system t2 should run. The included suites are t2B
%% %% and t2C. Included are also test cases test4, test1 and test7 in suite t2A. Note
%% %% that the test cases will be executed in the specified order.
%% %% * Lastly, all suites for systems t3 are to be completely skipped and this
%% %% should be explicitly noted in the log files.
%% %%
%% %% It is possible to specify initialization options for nodes defined in the test
%% %% specification. Currently, there are options to start the node and/or to evaluate
%% %% any function on the node. See the Automatic startup of the test target nodes
%% %% chapter for details.
%% %%
%% %% It is possible for the user to provide a test specification that includes (for
%% %% Common Test) unrecognizable terms. If this is desired, the -allow_user_terms
%% %% flag should be used when starting tests with ct_run. This forces Common Test to
%% %% ignore unrecognizable terms. Note that in this mode, Common Test is not able to
%% %% check the specification for errors as efficiently as if the scanner runs in
%% %% default mode. If ct:run_test/1 is used for starting the tests, the relaxed
%% %% scanner mode is enabled by means of the tuple: {allow_user_terms,true}
%% %% 6.9 Log files
%% %%
%% %% As the execution of the test suites proceed, events are logged in four different
%% %% ways:
%% %%
%% %% * Text to the operator's console.
%% %% * Suite related information is sent to the major log file.
%% %% * Case related information is sent to the minor log file.
%% %% * The HTML overview log file gets updated with test results.
%% %% * A link to all runs executed from a certain directory is written in the log
%% %% named "all_runs.html" and direct links to all tests (the latest results) are
%% %% written to the top level "index.html".
%% %%
%% %% Typically the operator, who may run hundreds or thousands of test cases, doesn't
%% %% want to fill the console with details about, or printouts from, the specific
%% %% test cases. By default, the operator will only see:
%% %%
%% %% * A confirmation that the test has started and information about how many
%% %% test cases will be executed totally.
%% %% * A small note about each failed test case.
%% %% * A summary of all the run test cases.
%% %% * A confirmation that the test run is complete.
%% %% * Some special information like error reports and progress reports,
%% %% printouts written with erlang:display/1, or io:format/3 specifically addressed
%% %% to a receiver other than standard_io (e.g. the default group leader process
%% %% 'user').
%% %%
%% %% If/when the operator wants to dig deeper into the general results, or the result
%% %% of a specific test case, he should do so by following the links in the HTML
%% %% presentation and take a look in the major or minor log files. The
%% %% "all_runs.html" page is a practical starting point usually. It's located in
%% %% logdir and contains a link to each test run including a quick overview (date and
%% %% time, node name, number of tests, test names and test result totals).
%% %%
%% %% An "index.html" page is written for each test run (i.e. stored in the "ct_run"
%% %% directory tagged with node name, date and time). This file gives a short
%% %% overview of all individual tests performed in the same test run. The test names
%% %% follow this convention:
%% %%
%% %% * TopLevelDir.TestDir (all suites in TestDir executed)
%% %% * TopLevelDir.TestDir:suites (specific suites were executed)
%% %% * TopLevelDir.TestDir.Suite (all cases in Suite executed)
%% %% * TopLevelDir.TestDir.Suite:cases (specific test cases were executed)
%% %% * TopLevelDir.TestDir.Suite.Case (only Case was executed)
%% %%
%% %% On the test run index page there is a link to the Common Test Framework log file
%% %% in which information about imported configuration data and general test progress
%% %% is written. This log file is useful to get snapshot information about the test
%% %% run during execution. It can also be very helpful when analyzing test results or
%% %% debugging test suites.
%% %%
%% %% On the test run index page it is noted if a test has missing suites (i.e. suites
%% %% that Common Test has failed to compile). Names of the missing suites can be
%% %% found in the Common Test Framework log file.
%% %%
%% %% The major logfile shows a detailed report of the test run. It includes test
%% %% suite and test case names, execution time, the exact reason for failures etc.
%% %% The information is available in both a file with textual and with HTML
%% %% representation. The HTML file shows a summary which gives a good overview of the
%% %% test run. It also has links to each individual test case log file for quick
%% %% viewing with an HTML browser.
%% %%
%% %% The minor log file contain full details of every single test case, each one in a
%% %% separate file. This way the files should be easy to compare with previous test
%% %% runs, even if the set of test cases change.
%% %%
%% %% Which information goes where is user configurable via the test server
%% %% controller. Three threshold values determine what comes out on screen, and in
%% %% the major or minor log files. See the OTP Test Server manual for information.
%% %% The contents that goes to the HTML log file is fixed however and cannot be
%% %% altered.
%% %%
%% %% The log files are written continously during a test run and links are always
%% %% created initially when a test starts. This makes it possible to follow test
%% %% progress simply by refreshing pages in the HTML browser. Statistics totals are
%% %% not presented until a test is complete however.
%% %% Log options
%% %%
%% %% With the logopts start flag, it's possible to specify options that modify some
%% %% aspects of the logging behaviour. Currently, the following options are
%% %% available:
%% %%
%% %% * no_src
%% %% * no_nl
%% %%
%% %% With no_src, the html version of the test suite source code will not be
%% %% generated during the test run (and consequently not be available in the log file
%% %% system).
%% %%
%% %% With no_nl, Common Test will not add a newline character (\n) to the end of an
%% %% output string that it receives from a call to e.g. io:format/2, and which it
%% %% prints to the test case log.
%% %%
%% %% For example, if a test is started with:
%% %%
%% %% $ ct_run -suite my_SUITE -logopts no_src
%% %%
%% %% then printouts during the test made by successive calls to io:format("x"), will
%% %% appear in the test case log as:
%% %%
%% %% xxx
%% %%
%% %% instead of each x printed on a new line, which is the default behaviour.