Ultimate Test Utility

Ⅰ. 插件描述

A simple, accessible testing plugin for use with Vim 7.0+.

Ⅱ. 基本信息

创建日期：  2013-09-12
使用用户：  2
Github星：  4
插件作者：  Kevin Biskar

Ⅲ. 安装方法

使用Vundle管理器安装

在你的.vimrc下添加: 
Plugin 'biskark/ultimate-test-utility'
… 然后在Vim中运行以下命令: 
:source %
:PluginInstall

对于Vundle版本 < 0.10.2，请用上面的Bundle替换Plugin。

使用NeoBundle管理器安装

在你的.vimrc下添加: 
NeoBundle 'biskark/ultimate-test-utility'
… 然后在Vim中运行以下命令: 
:source %
:NeoBundleInstall

使用VimPlug管理器安装

在你的.vimrc下添加: 
Plug 'biskark/ultimate-test-utility'
… 然后在Vim中运行以下命令: 
:source %
:PlugInstall

使用Pathogen管理器安装

在终端中运行以下命令: 
cd ~/.vim/bundle
git clone https://github.com/biskark/vim-ultimate-test-utility

Ⅳ. 文档说明

Vim Ultimate Test Utility

Introduction

Ultimate-Test-Utility is the only tool you'll need for creating and running
unit tests in Vim. Unit testing has been notoriously hard for Vim-script and
the plugins that do exist often require languages, are difficult to
setup/configure/use, have bugs, or have incomplete documentation.

Ultimate-Test-Utility is maintained solely by Kevin Biskar and is constantly
expanding it's features. If you have any concerns, feature requests, or have
found any undocumented bugs, please notify him (me).

Ultimate-Test-Utility can also be found at
vim.org, but this is
not recommended as that won't be as current this repository.

Features and Requirements

  • Ultimate-Test-Utility is written solely in vimscript so compilation
    requirements are minimal.
- However, Ultimate-Test-Utility must be run in a Vim 7.0 or higher and
  some functions may require read/write access to files you may be using.
  Of course, use those functions with care, though the instances in which
  these precautions are necessary should be obvious; Ultimate-Test-Utility
  does NOT want to harm your work, it's here to help.

- Also, Ultimate-Test-Utility has included some example functions that
  may not work correctly on your OS. This should not be an issue when
  using this plugin to write your own vimscript code, only when trying the
  demos. In the future, these examples may be updated to be compatible with
  as many OSes and setups as possible.
  • Ultimate-Test-Utility provides several levels of reporting so your reports
    are as detailed or minimal as you'd like.
  • Ultimate-Test-Utility allows for easy installation through Pathogen, Vundle,
    or other compatible plugin managers. If you're reading this, it's likely

installed correctly already, but if not consult the README.md file.

  • Ultimate-Test-Utility is, like most vim plugins, open-source! Please feel
    free to distribute, contribute, modify or spread the word.

Installation

Installation is quite simple and does not require any other plugins or
languages to be installed.

Pathogen

cd ~/.vim/bundle
git clone https://github.com/biskark/vim-ultimate-test-utility.git

That's enough to get the latest stable release, but if you'd like to keep
up-to-date with the latest developments, continue with...

cd ~/.vim/bundle/vim-ultimate-test-utility/
git checkout latest

From archive

  1. download archive
  2. unzip archive into .vim

Usage

Ultimate-Test-Utility has a straightforward API, and is not unlike the Perl
testing suite. However, Vim's lack of out-of-box OOP and our decision to
maximize compatibility (we don't use other languages or CLI testing
software) means that Ultimate-Test-Utility does have a mild
learning-curve, but should be easy to pick for both testing veterans and
first-timers alike.

  • The first section, Ultimate-Test-Utility-Quick-Start, provides a quick
  1. of how to use this plugin.
  • The second section, Ultimate-Test-Utility-Basic-Examples, shows the most
  1. usage and should be enough to get you started on working on your own

projects.

  • The third section, Ultimate-Test-Utility-Basic-Examples, explores more
  1. usage cases and explains the interface to some of the less intuitive

functions.

The more complex examples are available in examples.vim.

Quick Start Guide

Ultimate-Test-Utility breaks down tests into two categories. Individual
Assertion tests (referred to as sub-tests) which check for one specific thing,
and groupings of related sub-tests (collectively referred to as a test).
It's often natural to have sub-tests in a full test address different aspects
and edge cases of a particular function or key mapping. Thus a plugin with 4
functions and 1 keymapping may have 5 total "tests", with each "test"
consisting of up to dozens of "sub-tests."

The basic workflow is simple.

  1. You MUST call UltiTestStart to tell vim that you are going into
    "test-mode".
  2. You then call as many UltiAssert... sub-tests as you need.
  3. You then call UltiTestStop to tell vim that a particular test is over.
  4. You may then call UltiTestReport which will give a report of varying
    detail about the success and failure of your sub-tests. Additionally, this

will report the overall success of the entire test. The degree of detail
reported is dependent on the value of g:ulti_test_verbose. See
Ultimate-Test-Utility-Config for more details.

  1. If you wish to execute more tests (ie. more groupds of sub-tests), you
    should call UltiTestReset.
  2. Repeat steps 1-6 as needed.
  3. You may then call UltiTestFinalSummary to give a simple summary of all
    the "tests" you have run (since you either loaded the plugin or called

UltiTestResetAll, see next step for details).

  1. If you wish to run another suite of "tests", say for a different plugin or
    file, call UltiTestResetAll.
  2. Repeat steps 1-8 as needed.

The next section shows how to setup and run one full "test" on an example
function.

Basic Examples

Special Note: It's good practice to write your tests before you write the code
you're testing, but for the purpose of this tutorial, we'll ignore that.

Example Multiply function, takes two arguments and returns the quotient.

function! Multiply(numerator, denominator)
    return a:numerator * a:denominator
endfunction

Example test function that examines some basic-use cases and possible edge

  1. Comments have been sprinkled in to explain a little of the API.
function! MultiplyTestOne()
    call UltiTestStart()

    call UltiAssertTrue('', Multiply(2, 2) == 4, 'true')

    call UltiAssertTrue("Testing 1 times 1 isn't 6",
                \ Multiply(1, 1) == 6,
                \ 'false')

    " We're deliberately making an incorrect test here to see what happens
    call UltiAssertTrue("Testing 5 times -2 isn't -10",
                \ Multiply(5, -2) == -10,
                \ 'false')

    call UltiAssertEquals('Testing Multiply(2.0, 3) == Multiply(3.0, 2)',
                \ Multiply(2.0, 3),
                \ Multiply(3.0, 2),
                \ 'true')

    call UltiTestStop()
    call UltiTestReport()
endfunction

" Now to run the test
call MultiplyTestOne()

This should give something like the following as output:

    Sub-test 3: Failed, Testing 5 times -2 isn't -10
    Test 1 Results:
        Expected: Not specified, Passed: 3, Failed: 1, Skipped: 0

Yay, it works, let's review what we did.

The calls to UltiTestStart, UltiTestStop, and UltiTestReport were covered in
the previous section and are straightforward. However, we then called the
actual sub-tests themselves.

Each sub-test available starts with 'UltiAssert'.
Each UltiAssert function has a slightly different argument signature but there
are some similarities in usage.

  1. The first argument is a string describing the purpose of the test. Notice
    this message was displayed when sub-test 3 failed. Descriptive messages

make pin-pointing problems quick, but you may provide an empty string like
we did for the first sub-test.

  1. Next are the arguments unique to the assertion test being run.
  2. The last argument is optional, and can only be the string 'skip' to
    indicate that you don't want to run the test at this time. Notice that we

did not use this feature in any of the tests we just ran.

  1. The argument before 'skip' (aka the last argument if you don't
    provide 'skip'), must be either 'true' or 'false', indicating if

the Assertion is expected to be true or false.

Essentially, each simple UltiAssert... looks like this...

    call UltiAssertSimple(description, ..., expected_outcome)  " or
    call UltiAssertSimple(description, ..., expected_outcome, 'skip')

where the '...' is unique to whatever test you're using.

Note: there is no test called UltiAssertSimple or UltiAssert.... These are
place holder names.

Notice that in MultiplyTestOne we used UltiAssertTrue, which took one unique
argument (the result of the Multiply function call) and UltiAssertEquals which
took two unique arguments (the results of two Multiply function calls). As you
may have guessed, UltiAssertTrue simply evaluates the truthiness of its one
unique argument, and UltiAssertEquals compares its two unique arguments to

  1. Most sub-tests can be written with either one of these. The
  2. of both of them, and the more specific assertions explained in the

next section, is merely a convenience. You should use whichever sub-tests feel
the most natural.

Next Steps:
Yay, it works ... but it could be better. Ignoring that our function is
trivial, we deliberately made a mistake, and the usefulness of these tests is
questionable, we can still improve our test function.

  1. If we run our test again, it now says we're on Test 2 even though this
    is the same test we just ran.
  2. 'Test 1' isn't a horribly informative name in the first place.
  3. Though moot in this case, how do we know we tested everything we
    intended? Though careful coding can eliminate this problem, in tests with

many sub-tests, especially in more complicated plugins, we may want an
extra level of confidence.

It turns out we can fix all of these problems with minimal effort.

function! MultiplyTestTwo()
    call UltiTestReset()
    call UltiTestStart('Testing the Multiply Function', 4)

    call UltiAssertTrue('', Multiply(2, 2) == 4, 'true')

    call UltiAssertTrue("Testing 1 times 1 isn't 6",
                \ Multiply(1, 1) == 6,
                \ 'false')

    " We're deliberately making an incorrect test here to see what happens
    call UltiAssertTrue("Testing 5 times -2 isn't -10",
                \ Multiply(5, -2) == -10,
                \ 'false')

    call UltiAssertEquals('Testing Multiply(2.0, 3) == Multiply(3.0, 2)',
                \ Multiply(2.0, 3),
                \ Multiply(3.0, 2),
                \ 'true',
                \ 'skip')

    call UltiTestStop()
    call UltiTestReport()
endfunction

" Now to run the test
call MultiplyTestTwo()

This should give something like the following as output:

Sub-test 3: Failed, Testing 5 times -2 isn't -10
Test 1 'Testing the Multiply Function' Results:
    Expected: 4, Passed: 2, Failed: 1, Skipped: 1

Two quick points, and we can move on to more advanced usage.

  1. The call to UltiTestReset tells vim to reset our sub-test counters so
    that we could have been placed at the end of the function instead of the

    1. However, as we'll see, this call should usually be made outside
    2. any one test, and rather after or before running a suite of tests.
  2. The call to UltiTestStart can take two optional arguments in any order.
    It can take a string that should describe how the sub-tests are related,

and an integer that tells the test how many sub-tests to expect.

  1. The last sub-test has the 'skip' argument added, just to show how to use

    1. Note that adding 'skip' only prevents the result from being factored in
    2. testing facility, it won't prevent any errors in the other arguments

from causing problems.

Extended Examples

Let's broaden our look at this plugin be testing some more complicated
plugins/functions and integrating several tests into one suite. Assume all
references to MultiplyTest() refer to the MultiplyTestTwo() function from the
previous section.

Special Note: It's good practice to write your tests before you write the code
you're testing, but for the purpose of this tutorial, we'll ignore that.

First let's write two more functions and add a keymapping for one of them.

The first is a slightly trickier function that echos the quotient of two
numbers rather than merely returns them. Also, divide by zero exceptions may
be thrown and we'll show how to deal with that.

function! Divide(numerator, denominator)
    if a:denominator == 0
        throw "Divide by zero"
    endif
    echo (a:numerator / a:denominator)
endfunction

The next is a vim function that inserts the word "Hello" into the current

  1. For convenience, we'll also add a helper function that adds a
  2. for this function.
function! InsertHello()
    execute 'normal! iHello'
endfunction

function! NewMapping()
    nnoremap <silent> <leader>S :call InsertHello()<cr>
endfunction

Ok, now let's write those tests.

function! TestDivide()
    call UltiTestStart("Divide Tests", 5)

    " Output tests
    call UltiAssertInOutput('6 / 2 == 3', 'Divide', [6, 2], '3', 'true')
    call UltiAssertInOutput("1 / 3 == 0 ('Testing Int Division')",
                \ 'Divide', [1, 3], '6', 'false')

    " Exception tests
    call UltiAssertException("Throws divide by zero error",
                \ 'Divide by zero', 'Divide', [4, 0], 'true')
    call UltiAssertException("Doesn't throw divide by zero error",
                \ 'Divide by zero', 'Divide', [4, 2], 'false')
    " For variety, use a variable for the function name
    let l:fx = 'Divide'
    call UltiAssertException("Complains about arguments",
                \ 'Not enough arguments', l:fx, [], 'true')

    call UltiTestStop()
    call UltiTestReport()
endfunction

Notes:
UltiAssertInOuput and UltiAssertException both have more complicated
interfaces but share similarities with the previous UltiAssert functions
and with each other.

  1. Like the others, the first argument is a descriptive name, and the last
    two arguments are the expection (either "true" or "false") and an

optional "skip" string to indicate the test is skipped.

  1. The second argument is a variable or raw string with the name of the
    function to be tested. The name should NOT have parentheses or any

arguments.

  1. The third argument is a List of all arguments you would like to be
    passed to the function you're testing. If no arguments are to be passed,

you must provide an empty list '[]'.

  1. The fourth argument is a string or regex specific to the UltiAssert test
    you're using. For UltiAsserException, it should be the exception you

expect to be thrown. For UltiAssertInOutput it should be the string
you expect to be in the output.

In summary, each simple UltiAssert... looks like this...

call UltiAssertSimple(desc, ..., expected_outcome)  " or
call UltiAssertSimple(desc, ..., expected_outcome, 'skip')

where the '...' is unique to whatever test you're using.

Each complex UltiAssert... looks like this.

call UltiAssertComplex(desc, fx, arguments, string, expected_outcome)  " or
call UltiAssertComplex(desc, fx, arguments, string, expected_outcome, 'skip')

Note: there is no test called UltiAssertSimple or UltiAssert... or
UltiAssertComplex. These are place holder names only.

This next test is a bit of a cheat in that it tests the InsertHello() function
via the keymapping where these tests should probably be separate. Also, the
use of the mapleader variable can be troublesome because the mapping is always
whatever the value of the leader key at the time it was set. If this doesn't
make sense to you, just know to be careful if you use it.

function! TestInsertHello()
    call UltiTestStart('Testing Command Mapping and Text Insertion', 3)

    if exists('mapleader')
        let leader = mapleader
    else
        let leader = '\'
    endif

    " Make the new mapping
    call NewMapping()

    let l:fx = 'Hello'
    call UltiAssertInBuffer('Testing for "Hello" absence before execute',
                \ 'Hello', 'false')

    execute 'normal! ' . leader . 'S'
    call UltiAssertInBuffer('Testing for "Hello" absence after normal!',
                \ 'Hello', 'false')

    execute 'normal ' . leader . 'S'
    call UltiAssertInBuffer('Testing for "Hello World" presence after normal',
                \ 'Hello', 'true')

    " Clean up the text that was just inserted. Not necessary for the
    " test though.
    execute "s/Hello//"

    call UltiTestStop()
    call UltiTestReport()
endfunction

Notes

  1. We specified there should be 3 sub-tests run and given the test a
    description.
  2. We have not called UltiTestReset yet.
  3. Now that we're executing more complicated functions, we need
    to be careful that we're not polluting the current buffer with side-effects

from our testing. In particular beware of testing UltiAssertInBuffer
sub-tests in the buffer where you wrote the test. See if you can figure out
why.

  1. UltiAssertInBuffer has an almost identical interface to UltiAssertTrue.

Finally, let's organize these two tests into a full suite with the
MultiplyTest from before. Take out the call to UltiTestReset in the
MultiplyTest.

function! AllTests()
    call UltiTestReset()         " Clears residual junk

    call MultiplyTest()
    call TestDivide()
    call TestInsertHello()

    call UltiTestFinalSummary()  " Brief summary of all three tests
    call UltiTestResetAll()      " Clears everything
endfunction

Now to call the whole thing...

call AllTests()

Remember that because our TestInsertHello() tests for the presence and absence
of the string 'Hello' in the buffer it's executed in, it's best to run these
tests in an empty buffer. So it's probably best to source these functions,
switch to a new buffer, and then call AllTests().

You can change the output by setting one of these three values before you call
the function.

let g:ulti_test_verbose = 0    " or
let g:ulti_test_verbose = 1    " or
let g:ulti_test_verbose = 2

Assuming you left it at 1, you should get ...

Sub-test 3: Failed, Testing 5 times -2 isn't -10
Test 1 Results:
    Expected: 4, Passed: 3, Failed: 1, Skipped: 0

Test 2 'Divide Test' Results
    Expected: 5, Passed: 5, Failed: 0, Skipped: 0

Test 3 'Testing Command Mapping and Text Insertion' Results
    Expected: 3, Passed: 3, Failed: 0, Skipped: 0

1 test(s) failed.

添加新评论