Testerrific tries to take the headache out of site testing by making things simple and concise.
By far the easiest syntax of any testing suite. Write tests as simply as tt.test("a < 4")
Test anything that resolves truthy or falsey, no need to adjust your code.
Even though it's so simple, you can customize tests in a number of ways to create exactly the scenarios you need to check.
You can disable tests, pause execution, or even run a specific test on demand—all without having to go back to your code to make changes.
Change the Testerrific data object (ie. 'tt') and the UI will update automatically, which helps a lot with on-the-fly tweaks and the trial-and-error phase of writing tests.
Whether you're waiting for the server, the DOM, even on user input, Testerrific has things like auto-retry, conditional checks, and promise-friendly inputs to make sure tests run when they should.
A whole set of functions and events to be able to track and adjust how your tests affect or respond to just about any situation on your site.
Testerrific weighs only 8.5kb gzipped (5.7kb JS, 2.8kb CSS) and doesn't require any other libraries.
Just link to the Testerrific JS and CSS files...
<!-- Download and link to the files locally: -->
<script src="testerrific.min.js"></script>
<link rel="Stylesheet" href="testerrific.min.css" type="text/css" />
<!-- OR use link to the files on jsdelivr CDN: -->
<script src="https://cdn.jsdelivr.net/gh/thomhines/testerrific@master/testerrific.min.js"></script>
<link rel="Stylesheet" href="https://cdn.jsdelivr.net/gh/thomhines/testerrific@master/testerrific.min.css" type="text/css" />
and then add some tests in your javascript.
Tests are organized into test groups, and every group is saved as an array in tt.groups. Testerrific uses a "reactive" UI, meaning that changes made to the data saved in the tt objects are shown instantly in the interface. Adding or changing a test will automatically make that test show up in the tests list.
To start adding tests, simply run tt.test() to add a test to your list
tt.group('First Test Group');
tt.test('1 + 1 == 2');
tt.test("Is browser window tall enough?", 'window.innerHeight > 200');
tt.test("Is it after 8am?", () => { return (new Date()).getHours() >= 8 });
The group() function will create a new test group. Every test written after that will be in that test group until another test group is created.
label: (string) text shown as title for test group
options: (object) that can contain the following values:
skip: (boolean) whether to skip this test or not
solo: (boolean) skip all other tests that are not also flagged as "solo"
collapse: (boolean) whether tests are shown or not when page is loaded
beforeEach: (function/promise) function that gets run before each test in group
afterEach: (function/promise) function that gets run after each test in group
To create a new test group:
tt.group("Simple Test Group");
You can also add settings to a group to modify how it loads and functions.
tt.group("Skipped Test Group", {
skip: 1
});
tt.group("Another Test Group", {
beforeEach: () => { /* Run this code before each test */ }
});
The test() function will create a new test (string, function, or promise) and add it to the end of the last test group. If you haven't defined a test group yet, Testerrific will make one for you.
label: (string) text shown as title for test. If no label is given and "check" value is a string, the "check" string will be used as the label.
check: (string) that evaluates as truthy/falsey.
OR
(function) that returns truthy/falsey.
OR
(promise) that resolves truthy/falsey.
options: (object) that can contain the following values:
skip: (boolean) whether to skip this test or not
solo: (boolean) skip all other tests that are not also flagged as "solo"
wait_for: (string) boolean javascript condition that must return truthy before starting test
max_time: (integer) the maximum amount of time a test should be allowed to run before being failed
run_if: (string that evals to truthy/falsey) Conditional that determines if test should be run
message: (string) message shown in UI for the duration of the test, usually instructions for tests that need manual intervention
group: (string or integer) the title or the group index of the group the test should be added to
position: (integer) position within a group the test should be inserted (starting from 0)
before: (function/promise) gets run before test. If a promise, test will wait for promise to resolve before beginning.
after: (function/promise) gets run after test. If a promise, test will wait for promise to resolve before moving to next test.
Test if a string evaluates as truthy or falsey
tt.test("1 > 0");
You can also add a label to make your test easier to read or understand.
tt.test("Is '1' a positive number?", "1 > 0");
tt.test("Is today Thursday?", () => {
return (new Date()).getDay() == 4;
});
Tests that return a promise will run until they resolve with a truthy or falsey value.
tt.test("Test as Promise example", () => {
return new Promise(function(resolve, reject) {
setTimeout(() => {
resolve(true);
}, 500);
})
});
Tests that can't be accomplished or validated programatically can be
tt.manual_test("'manual test' example", "Clicking buttons below will cause test to pass or fail")
Disable this test. Skipped tests are only disabled when the tests are first loaded. You can always re-enable a 'skipped' test or run it manually.
tt.test("'skip' example", "4 == 4", {
skip: true
});
Skip all other tests/groups. You can set multiple tests/groups as "solo".
tt.test("'solo' example", "4 == 4", {
solo: true
});
This effect will only happen when Testerrific is first loaded, or if you run skip_non_only_tests().
tt.skip_non_only_tests()
Wait to run test until a given condition is met. The test will wait up until the number of milliseconds set in tt.wait_for_timeout for statement to return truthy.
tt.run('Fade in .test_element', () => {
$('.test_element').fadeIn(3000)
});
// Wait until .test_element is fully visible before starting test
tt.test('wait_for example', '4 == 4', {
wait_for: "$('.test_element').is(':visible') && $('.test_element').css('opacity') == '1'",
after: () => {
$('.test_element').fadeOut(1000)
}
});
By default, Testerrific will retry a test multiple times until it returns truthy, in case there are asynchronous factors that a test might be waiting on. This setting controls the amount of time (integer, in ms) to keep trying a test. Set to '0' to disable auto-retry.
tt.test("'max_time' example (wait 3 sec before giving up)", 'false', {
max_time: 3000
});
A code-test check to determine if test should be run.
b = 3;
tt.test("'run_if' example (run_if == false)", '1 > 0', {
run_if: 'b == 2'
});
tt.test("'run_if' example (run_if == true)", '1 > 0', {
run_if: 'b == 3'
});
Display a message while test is running. This can be helpful to give status updates or instructions to user for manual intervention.
tt.test("'message' example", 'a != 3', {
message: 'Just saying hello!',
max_time: 3000 // max_time increased to allow message to appear for longer
});
Group value can either be the index (integer, starting from 0) or title (string) of an existing test group. Position is the where the test should be inserted within the group (integer, starting from 0). If no position is given, the test will be added to the end of the selected group.
// Add this test to the 4th spot of the 4th test group
tt.test('Add test to specific group example', '1 > 0', {
group: 'Simple Test Group'
});
tt.test('Add test at Group and Position index example', '1 > 0', {
group: 3,
position: 3
});
(function/promise) gets run before/after test
a = 0
tt.test("'before()' example", 'a == 1', {
before: () => {
a = 1
}
});
Run an arbitrary function as an item in the tests list.
label: (string) text shown as title for test group. If no label is given and "check" value is a string, the check value will be used as the label fn: (function/promise) Function to be run. If a promise is given, tests will wait for promise to resolve before continuing. options: (object) Options to affect how function is run. See options for test() function above for list of available options.
tt.run("change a to -3", () => { a = -3 });
tt.test("Is 'a' still positive?", "a > 0");
Wait a specified amount of time (in ms) before moving to next test.
tt.wait(3000);
Pause tests at this point. User will need to manually click 'Resume' in order for tests to continue.
tt.pause();
Because Testerrific's UI is reactive, you can manipulate a number of aspects by changing properties of the tt object.
The amount of time (in ms) to keep trying a test until it returns truthy. This applies to all tests unless a test overwrites its default max_time setting.
tt.max_time = 1000
Whether or not the tests panel is visible or shown.
To start with the Testerrific panel closed, for instance, simply add this to your code that gets run on page load:
tt.visible = false;
Run all of the unskipped tests. If a group_index is given, it will only run the tests in that group. If no index is given, all tests will be run starting from the first group.
tt.start_tests();
Pause execution of tests.
Note: the tests will only get paused at the conclusion of the current test
tt.pause_tests();
Resume execution of tests.
tt.resume_tests();
Run specific test. Both group_index and test_index are integers of the position of which group and test to run (starting from 0).
tt.run_test(0,2);
Set specific test as 'passed'. Both group_index and test_index are integers of the position of which group and test to run (starting from 0). If either index is blank, pass_test() will assume the group and/or test currently being run.
You can use this to pass tests triggered by outside actions, such as...
tt.pass_test(0,2);
Set specific test as 'failed'. Both group_index and test_index are integers of the position of which group and test to run (starting from 0). If either index is blank, fail_test() will assume the group and/or test currently being run.
You can use this to fail tests triggered by outside actions, such as...
tt.fail_test(0,2);
Set specific test as 'skipped'. Both group_index and test_index are integers of the position of which group and test to run (starting from 0). If either index is blank, skip_test() will assume the group and/or test currently being run.
You can use this to skip tests triggered by outside actions, such as...
tt.skip_test(0,2);
Quit execution of tests, cancelling any test that is currently being run.
tt.finish_tests();
Reset all results and timers for all tests.
tt.reset_tests();
Display text 'message' (string) for the number of milliseconds set in time_limit. If time_limit is not set, it will display the message indefinitely or until another message is shown.
tt.alert('You obviously love to press the buttons', 3000);
Open tests panel if closed, close it if it's open.
tt.toggle_tests_panel();
Mark all groups and tests as enabled (ie. not 'skipped').
tt.enable_all_groups();
Mark all groups and tests as disabled (ie. 'skipped').
tt.disable_all_groups();
Visually collapse all groups in the tests panel. Tests within a collapsed group will still run even if they aren't visible.
tt.collapse_all_groups();
Visually expand all groups in the tests panel.
tt.expand_all_groups();
Toggle whether the tests within a specific test group are visible.
tt.toggle_view_group(0);
Toggle whether the tests within a specific test group are enabled (ie. not 'skipped') or not.
tt.toggle_skip_group(0);
Toggle whether a specific test within a specific test group are enabled (ie. not 'skipped') or not.
tt.toggle_skip_test(0, 2);
Returns the number of tests that have a certain result type for a specific test group. Result types can be 'all', 'run', 'passed', 'failed', 'skipped', or 'error'. If not type is given, totals() will assume 'all'. If not group_index is given, totals() will return a count for all test groups.
alert(`
Total number of tests:
${tt.totals()}
Total number 'passed' tests:
${tt.totals('passed')}
Total number of 'skipped' tests in the 2nd test group:
${tt.totals('skipped', 1)}
`);
You can add a "pause" between any tests without updating your tests or refreshing the page by clicking the faded pause button to the left of each test in the Testerrific panel.
Each group and test is an element in an array (tt.groups). To start with all of the test groups closed, just loop through all of your groups and set their 'collapse' property to true like this:
tt.groups.forEach((group) => { group.collapse = true });
To change a test programmatically, simply change the info for that test in the tt.groups array. This code will set the first test in the first group to be skipped:
tt.groups[0].tests[0].skip = true;
Add this snippet to your code to make running tests even faster. You can try it out now by pressing Cmd-G or Ctrl-G.
document.onkeydown = function(event) {
if(event.metaKey = true && event.key == 'g') {
event.preventDefault();
event.stopPropagation();
if(tt.paused) tt.resume_tests();
else if(tt.running) tt.pause_tests();
else tt.start_tests();
}
}
There are times when you may want to run a set of tests that are very similar to another set of tests without having to duplicate everything. You can copy and modify a test group simply by cloning it in the tt.groups array and altering the various properties
There are however, some things to note. Testerrific stores all of its data in a Proxy, which works more as a data reference than a stand-alone object variable. In order to actually duplicate a group or a test into a new group/test, you have to perform a deep clone of all data within that group/test. The easiest way to do this is with something like Lodash's cloneDeep() function.
let cloned_group = _.cloneDeep(tt.groups[0]);
cloned_group.label = "This is a copy of the first test group";
tt.groups.push(cloned_group);