patch 9.0.1627: no generic mechanism to test syntax plugins
Problem: No generic mechanism to test syntax plugins.
Solution: Add a syntax plugin test mechanism, using screendumps. Add a
simple test for "c".
This commit is contained in:
97
runtime/syntax/testdir/README.txt
Normal file
97
runtime/syntax/testdir/README.txt
Normal file
@ -0,0 +1,97 @@
|
||||
Tests for syntax highlighting plugins
|
||||
=====================================
|
||||
|
||||
Summary: Files in the "input" directory are edited by Vim with syntax
|
||||
highlighting enabled. Screendumps are generated and compared with the
|
||||
expected screendumps in the "dumps" directory. This will uncover any
|
||||
character attributes that differ.
|
||||
|
||||
Without any further setup a screendump is made at the top of the file (using
|
||||
_00.dump) and another one at the end of the file (using _99.dump). The dumps
|
||||
are normally 20 screen lines tall.
|
||||
|
||||
When the screendumps are OK an empty "done/{name}" file is created. This
|
||||
avoids running the test again until "make clean" is used. Thus you can run
|
||||
"make test", see one test fail, try to fix the problem, then run "make test"
|
||||
again to only repeat the failing test.
|
||||
|
||||
When a screendump differs it is stored in the "failed" directory. This allows
|
||||
for comparing it with the expected screendump, using a command like:
|
||||
|
||||
let fname = '{name}_99.dump'
|
||||
call term_dumpdiff('failed/' .. fname, 'dumps/' .. fname)
|
||||
|
||||
|
||||
Creating a syntax plugin test
|
||||
-----------------------------
|
||||
|
||||
Create a source file in the language you want to test in the "input"
|
||||
directory. Make sure to include some interesting constructs with complicated
|
||||
highlighting.
|
||||
|
||||
Use the filetype name as the base and a file name extension matching the
|
||||
filetype. Let's use Java as an example. The file would then be
|
||||
"input/java.java".
|
||||
|
||||
If there is no further setup required, you can now run the tests:
|
||||
make test
|
||||
|
||||
The first time this will fail with an error for a missing screendump. The
|
||||
newly created screendumps will be "failed/java_00.dump",
|
||||
"failed/java_01.dump", etc. You can inspect each with:
|
||||
|
||||
call term_dumpload('failed/java_00.dump')
|
||||
call term_dumpload('failed/java_01.dump')
|
||||
...
|
||||
call term_dumpload('failed/java_99.dump')
|
||||
|
||||
If they look OK, move them to the "dumps" directory:
|
||||
|
||||
:!mv failed/java_00.dump dumps
|
||||
:!mv failed/java_01.dump dumps
|
||||
...
|
||||
:!mv failed/java_99.dump dumps
|
||||
|
||||
If you now run the test again, it will succeed.
|
||||
|
||||
|
||||
Adjusting a syntax plugin test
|
||||
------------------------------
|
||||
|
||||
If you make changes to the syntax plugin, you should add code to the input
|
||||
file to see the effect of these changes. So that the effect of the changes
|
||||
are covered by the test. You can follow these steps:
|
||||
|
||||
1. Edit the syntax plugin somewhere in your personal setup. Use a file
|
||||
somewhere to try out the changes.
|
||||
2. Go to the directory where you have the Vim code checked out and replace the
|
||||
syntax plugin. Run the tests: "make test". Usually the tests will still
|
||||
pass, but if you fixed syntax highlighting that was already visible in the
|
||||
input file, carefully check that the changes in the screendump are
|
||||
intentional:
|
||||
let fname = '{name}_99.dump'
|
||||
call term_dumpdiff('failed/' .. fname, 'dumps/' .. fname)
|
||||
Fix the syntax plugin until the result is good.
|
||||
2. Edit the input file for your language to add the items you have improved.
|
||||
(TODO: how to add another screendump?).
|
||||
Run the tests and you should get failures. Like with the previous step,
|
||||
carefully check that the new screendumps in the "failed" directory are
|
||||
good. Update the syntax plugin and the input file until the highlighting
|
||||
is good and you can see the effect of the syntax plugin improvements. Then
|
||||
move the screendumps from the "failed" to the "dumps" directory. Now "make
|
||||
test" should succeed.
|
||||
3. Prepare a pull request with the modified files:
|
||||
- syntax plugin: syntax/{name}.vim
|
||||
- test input file: syntax/testdir/input/{name}.{ext}
|
||||
- test dump files: syntax/testdir/dumps/{name}_99.dump
|
||||
|
||||
As an extra check you can temporarily put back the old syntax plugin and
|
||||
verify that the tests fail. Then you know your changes are covered by the
|
||||
test.
|
||||
|
||||
|
||||
|
||||
TODO: run test for one specific filetype
|
||||
|
||||
TODO: testing with various option values
|
||||
TODO: test syncing by jumping around
|
||||
Reference in New Issue
Block a user