runtime(syntax-tests): Facilitate the viewing of rendered screendumps

With the submitted "viewdumps.vim" script, a few manual
steps in typical workflows (see below) can be automated.
The updated "README.txt" contains additional information.

============================================================

Reviewing LOCAL failed syntax tests can be arranged as
follows:

1) Run tests and generate screendumps:
------------------------------------------------------------
cd /path/to/fork/runtime/syntax
make clean test
------------------------------------------------------------

2) Examine the screendumps from the "failed" directory:
------------------------------------------------------------
../../src/vim --clean -S testdir/viewdumps.vim
------------------------------------------------------------

============================================================

Reviewing UPLOADED failed syntax tests can be arranged as
follows (it can be further locally scripted):

1) Fetch an artifact with failed screendumps from
"github.com/vim/vim/actions/runs/A_ID/artifacts/B_ID".

2) Extract the archived files:
------------------------------------------------------------
unzip /tmp/failed-tests.zip -d /tmp
------------------------------------------------------------

3) Set up the "dumps" directory.  Create a symlink to
"/path/to/fork/dirs/dumps" in the extracted directories so
that term_dumpdiff() can be used.  (The lookup algorithm
resolves "dumps" for every loaded filename.  So, with
"/tmp/runtime/syntax/testdir/failed/*.dump" files passed
as script arguments, the algorithm will make the files in
"/tmp/runtime/syntax/testdir/dumps" queried.)
------------------------------------------------------------
cd /path/to/fork
ln -s $(pwd)/runtime/syntax/testdir/dumps \
  /tmp/runtime/syntax/testdir/dumps
------------------------------------------------------------

4) Examine the extracted screendumps:
------------------------------------------------------------
./src/vim --clean -S runtime/syntax/testdir/viewdumps.vim \
  /tmp/runtime/syntax/testdir/failed/*.dump
------------------------------------------------------------

5) Clean up:
------------------------------------------------------------
unlink /tmp/runtime/syntax/testdir/dumps
rm -rf /tmp/runtime
------------------------------------------------------------

============================================================

Reviewing SUBMITTED FOR PULL REQUEST syntax tests can be
arranged as follows (it can be further locally scripted):

1) List the fetched changeset and write the changed "dumps"
filenames to "/tmp/filelist":
------------------------------------------------------------
cd /path/to/fork
git switch prs/1234
git diff-index --relative=runtime/syntax/testdir/dumps/ \
  --name-only prs/1234~1 > /tmp/filelist
------------------------------------------------------------

2) Reconcile relative filepaths, and copy next-to-be-updated
"dumps" files in the "failed" directory (note the missing
new screendumps, if any):
------------------------------------------------------------
git switch master
cd runtime/syntax/testdir/dumps
cp -t ../failed $(cat /tmp/filelist)
------------------------------------------------------------

3) Remember about the introduced INVERTED relation between
"dumps" and "failed", i.e. the files to be committed are in
"dumps" already and their previous versions are in "failed";
therefore, copy the missing new screendumps from "dumps" to
"failed" (otherwise these won't be shown):
------------------------------------------------------------
git switch prs/1234
cp -t ../failed foo_10.dump foo_11.dump foo_12.dump
------------------------------------------------------------

4) Examine the screendumps from the "failed" directory (new
screendumps will be shown with no difference between their
versions):
------------------------------------------------------------
cd ..
../../../src/vim --clean -S viewdumps.vim
------------------------------------------------------------

closes: #15476

Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
This commit is contained in:
Aliaksei Budavei
2024-08-12 18:37:15 +02:00
committed by Christian Brabandt
parent 90e1fe4b76
commit d33afe12c6
6 changed files with 120 additions and 4 deletions

View File

@ -124,6 +124,31 @@ verify that the tests fail. Then you know your changes are covered by the
test.
Viewing generated screendumps
-----------------------------
You may also wish to look at the whole batch of failed screendumps after
running "make test". Source the "viewdumps.vim" script for this task:
[VIMRUNTIME=../..] \
../../src/vim --clean -S testdir/viewdumps.vim \
[testdir/dumps/java_*.dump ...]
By default, all screendumps found in the "failed" directory will be added to
the argument list and then the first one will be loaded. Loaded screendumps
that bear filenames of screendumps found in the "dumps" directory will be
rendering the contents of any such pair of files and the difference between
them (:help term_dumpdiff()); otherwise, they will be rendering own contents
(:help term_dumpload()). Remember to execute :edit when occasionally you see
raw file contents instead of rendered.
At any time, you can add, list, and abandon other screendumps:
:$argedit testdir/dumps/java_*.dump
:args
:qall
The listing of argument commands can be found under :help buffer-list.
TODO: run test for one specific filetype