Problem: Generating prototype files does not work on all platforms
Solution: Rework prototypes generation using python instead of cproto,
enable it in CI to test it for each PR (Hirohito Higashi).
closes: #18045
Signed-off-by: Hirohito Higashi <h.east.727@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: diff: w_topline may be invalidated
Solution: Update lnum in diff_set_topline()
(Yee Cheng Chin).
This can happen in ex_diffupdate() for certain edge cases which cause
the logic to now be wrong. This was also the root cause for #18437 where
Vim would crash due to a null pointer dereferencing (said pointer would
not be null under normal circumstances).
related: #18437closes: #18484
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: possible crash when calculating topline in diff.c
(youngmith)
Solution: Check for pointer being Null before accessing it
fixes: #18437
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: defaults: 'diffopt' option value can be improved
Solution: Update diffopt defaults to include "indent-heuristic" and
"inline:char" (Yee Cheng Chin)
The default diff options have not been updated much despite new
functionality having been added to Vim.
- indent-heurstic: This has been enabled by default in Git since
33de716387 in 2017. Given that Vim uses xdiff from Git, it makes sense
to track the default configuration from Git.
- inline:char: This turns on character-wise inline highlighting which is
generally much better than the default inline:simple. It has been
implemented since #16881 and we have not seen reports of any issues
with it, and it has received good feedbacks.
closes: #18255
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: diff: using diff anchors with hidden buffers fails silently
Solution: Give specific error message for diff anchors when using hidden
buffers (Yee Cheng Chin).
Diff anchors currently will fail to parse if a buffer used for diff'ing
is hidden. Previously it would just fail as the code assumes it would
not happen normally, but this is actually possible to do if `closeoff`
and `hideoff` are not set in diffopt. Git's default diff tool "vimdiff3"
also takes advantage of this.
This fix this properly would require the `{address}` parser to be
smarter about whether a particular address relies on window position or
not (e.g. the `'.` address requires an active window, but `'a` or `1234`
do not). Since hidden diff buffers seem relatively niche, just provide a
better error message / documentation for now. This could be improved
later if there's a demand for it.
related: #17615closes: #17904
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Crash when using inline diff mode
(Ilya Grigoriev)
Solution: Set tp_diffbuf to NULL when skipping a diff block
(Yee Cheng Chin).
Fix an array out of bounds crash when using diffopt+=inline:char when 4
or more buffers are being diff'ed. This happens when one of the blocks
is empty. The inline highlight logic skips using that buffer's block,
but when another buffer is used later and calls diff_read() to merge the
diff blocks together, it could erroneously consider the empty block's
diff info which has not been initialized, leaving to diff numbers that
are invalid. Later on the diff num is used without bounds checking which
leads to the crash.
Fix this by making sure to unset tp_diffbuf to NULL when we skip a
block, so diff_read() will not consider this buffer to be used within
inline diff. Also, add more bounds checking just to be safe.
closes: #17805
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: not possible to anchor specific lines in difff mode
Solution: Add support for the anchoring lines in diff mode using the
'diffanchor' option (Yee Cheng Chin).
Adds support for anchoring specific lines to each other while viewing a
diff. While lines are anchored, they are guaranteed to be aligned to
each other in a diff view, allowing the user to control and inform the
diff algorithm what the desired alignment is. Internally, this is done
by splitting up the buffer at each anchor and run the diff algorithm on
each split section separately, and then merge the results back for a
logically consistent diff result.
To do this, add a new "diffanchors" option that takes a list of
`{address}`, and a new "diffopt" option value "anchor". Each address
specified will be an anchor, and the user can choose to use any type of
address, including marks, line numbers, or pattern search. Anchors are
sorted by line number in each file, and it's possible to have multiple
anchors on the same line (this is useful when doing multi-buffer diff).
Update documentation to provide examples.
This is similar to Git diff's `--anchored` flag. Other diff tools like
Meld/Araxis Merge also have similar features (called "synchronization
points" or "synchronization links"). We are not using Git/Xdiff's
`--anchored` implementation here because it has a very limited API
(it requires usage of the Patience algorithm, and can only anchor
unique lines that are the same across both files).
Because the user could anchor anywhere, diff anchors could result in
adjacent diff blocks (one block is directly touching another without a
gap), if there is a change right above the anchor point. We don't want
to merge these diff blocks because we want to line up the change at the
anchor. Adjacent diff blocks were first allowed when linematch was
added, but the existing code had a lot of branched paths where
line-matched diff blocks were handled differently. As a part of this
change, refactor them to have a more unified code path that is
generalized enough to handle adjacent diff blocks correctly and without
needing to carve in exceptions all over the place.
closes: #17615
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: inconsistent range arg for :diffget/diffput
Solution: fix the range specification, place the cursor for :diffput and
:diffget consistently on the last line (Yee Cheng Chin)
Previously, `:<range>diffget` only allowed using 1 or above in the range
value, making it impossible to use the command for a diff block at the
beginning of the file. Fix the range specification so the user can now
use 0 to specify the space before the first line. This allows
`:0,$+1diffget` to work to retrieve all the changes from the other file
instead of missing the first diff block. Also do this for `:diffput`.
Also, make `:diffput` work more similar to `:diffget`. Make it so that
if the cursor is on the last line and a new line is inserted in the
other file, doing `:diffput` will select that diff block below the line,
just like `:diffget` would.
Also clean up the logic a little bit for edge cases and for handling
line matched diff blocks better.
closes: #17579
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Last diff folds not merged (after v8.1.1922)
Solution: loop over all windows in the current tabpage and update all
folds (Gary Johnson)
This commit fixes a bug where the last two folds of a diff are not
merged when the last difference between the two diff'd buffers is
resolved.
Normally, when two buffers are diff'd, folding is used to show only the
text that differs and to hide the text that is the same between the two
buffers. When a difference is resolved by making a block of text the
same in both buffers, the folds are updated to merge that block with the
folds above and below it into one closed fold.
That updating of the folds did not occur when the block of text was the
last diff block in the buffers.
The bug was introduced by this patch on August 24, 2019:
patch 8.1.1922: in diff mode global operations can be very slow
Problem: In diff mode global operations can be very slow.
Solution: Do not call diff_redraw() many times, call it once when
redrawing. And also don't update folds multiple times.
Unfortunately, folds were then not updated often enough.
The problem was fixed by adding a short loop to the ex_diffgetput()
function in diff.c to update all the folds in the current tab when the
last difference is removed.
A test for this was added to test_diffmode.vim. Two of the reference
screen dumps for another test in that file,
Test_diffget_diffput_linematch(), had to be changed to have all the
folds closed rather than to have the last diff block remain open.
closes: #17457
Signed-off-by: Gary Johnson <garyjohn@spocom.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: style: more wrong indentation
Solution: reformat a few more places
(Yegappan Lakshmanan)
closes: #17309
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Various typos in the code, redundant and strange use of
:execute in test_ins_complete.vim (after 9.1.1315).
Solution: Fix typos in the code and in the documentation, use the
executed command directly (zeertzjq).
closes: #17143
Co-authored-by: Christ van Willegen <cvwillegen@gmail.com>
Signed-off-by: zeertzjq <zeertzjq@outlook.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: inline word diff treats multibyte chars as word char
(after 9.1.1243)
Solution: treat all non-alphanumeric characters as non-word characters
(Yee Cheng Chin)
Previously inline word diff simply used Vim's definition of keyword to
determine what is a word, which leads to multi-byte character classes
such as emojis and CJK (Chinese/Japanese/Korean) characters all
classifying as word characters, leading to entire sentences being
grouped as a single word which does not provide meaningful information
in a diff highlight.
Fix this by treating all non-alphanumeric characters (with class number
above 2) as non-word characters, as there is usually no benefit in using
word diff on them. These include CJK characters, emojis, and also
subscript/superscript numbers. Meanwhile, multi-byte characters like
Cyrillic and Greek letters will still continue to considered as words.
Note that this is slightly inconsistent with how words are defined
elsewhere, as Vim usually considers any character with class >=2 to be
a "word".
related: #16881 (diff inline highlight)
closes: #17050
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Diff mode's inline highlighting is lackluster. It only
performs a line-by-line comparison, and calculates a single
shortest range within a line that could encompass all the
changes. In lines with multiple changes, or those that span
multiple lines, this approach tends to end up highlighting
much more than necessary.
Solution: Implement new inline highlighting modes by doing per-character
or per-word diff within the diff block, and highlight only the
relevant parts, add "inline:simple" to the defaults (which is
the old behaviour)
This change introduces a new diffopt option "inline:<type>". Setting to
"none" will disable all inline highlighting, "simple" (the default) will
use the old behavior, "char" / "word" will perform a character/word-wise
diff of the texts within each diff block and only highlight the
differences.
The new char/word inline diff only use the internal xdiff, and will
respect diff options such as algorithm choice, icase, and misc iwhite
options. indent-heuristics is always on to perform better sliding.
For character highlight, a post-process of the diff results is first
applied before we show the highlight. This is because a naive diff will
create a result with a lot of small diff chunks and gaps, due to the
repetitive nature of individual characters. The post-process is a
heuristic-based refinement that attempts to merge adjacent diff blocks
if they are separated by a short gap (1-3 characters), and can be
further tuned in the future for better results. This process results in
more characters than necessary being highlighted but overall less visual
noise.
For word highlight, always use first buffer's iskeyword definition.
Otherwise if each buffer has different iskeyword settings we would not
be able to group words properly.
The char/word diffing is always per-diff block, not per line, meaning
that changes that span multiple lines will show up correctly.
Added/removed newlines are not shown by default, but if the user has
'list' set (with "eol" listchar defined), the eol character will be be
highlighted correctly for the specific newline characters.
Also, add a new "DiffTextAdd" highlight group linked to "DiffText" by
default. It allows color schemes to use different colors for texts that
have been added within a line versus modified.
This doesn't interact with linematch perfectly currently. The linematch
feature splits up diff blocks into multiple smaller blocks for better
visual matching, which makes inline highlight less useful especially for
multi-line change (e.g. a line is broken into two lines). This could be
addressed in the future.
As a side change, this also removes the bounds checking introduced to
diff_read() as they were added to mask existing logic bugs that were
properly fixed in #16768.
closes: #16881
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Vim's diff block merging algorithm when doing a multi-file diff
is buggy when two different diff hunks overlap a single
existing diff block (after v9.1.0743)
Solution: fix a couple bugs in this logic:
1. Fix regression from v9.1.0743 where it's not correctly expanding the
2nd overlap correctly, where it always expands without taking into
account that this was always taken care of when the first overlap
happened. Instead, we should only grow the 2nd overlap if it overhangs
outside the existing diff block, and if we encounter a new overlapping
diff block (due to overlap chaining).
2. When we expand a diff block to match the hunk size on the orig side
(when handling the first overlap), we expand the same amount of lines
in the new side. This is not sound if there exists a second overlap
hunk that we haven't processed yet, and that hunk has different
number of lines in orig/new. Fix this by doing the corresponding
counter adjustment when handling 2nd/3rd/etc overlap by calculating
the difference in lines between orig and new side.
(Yee Cheng Chin)
closes: #16768
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: 'diffopt' "linematch" cannot be used with {n} less than 10
digits (after v9.1.1022)
Solution: Fix off-by-one error when checking for digit (zeertzjq)
closes: #16577
Signed-off-by: zeertzjq <zeertzjq@outlook.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: no sanitize check when running linematch
Solution: add sanitize check before applying the linematch algorithm,
similar to diff_find_change() (Jonathon)
closes: #16446
Signed-off-by: Jonathon <jonathonwhite@protonmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Coverity complains about dereferencing NULL pointer
Solution: Verify curdiff is not null before dereferencing it
closes: #16437
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: v9.1.0743 causes regression with diff mode
Solution: Fix the regression with overlapping regions
closes: #16454
Signed-off-by: Yukihiro Nakadaira <yukihiro.nakadaira@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: diff feature can be improved
Solution: include the linematch diff alignment algorithm
(Jonathon)
closes: #9661
Signed-off-by: Jonathon <jonathonwhite@protonmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: diff mode does not handle overlapping diffs correctly
Solution: correct the logic to handle overlapping blocks
(Yukihiro Nakadaira)
Vim merges overlapped diff blocks and it doesn't work expectedly
in some situation.
closes: #15735
Signed-off-by: Yukihiro Nakadaira <yukihiro.nakadaira@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: incorrect internal diff with an empty file
Solution: Set pointer to NULL, instead of using an empty line file
(Yukihiro Nakadaira)
When using internal diff, empty file is read as one empty line file.
So result differs from external diff.
closes: #15719
Signed-off-by: Yukihiro Nakadaira <yukihiro.nakadaira@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: More code can use ml_get_buf_len() instead of STRLEN().
Solution: Change more STRLEN() calls to ml_get_buf_len(). Also do not
set ml_line_textlen in ml_replace_len() if "has_props" is set,
because "len_arg" also includes the size of text properties in
that case. (zeertzjq)
closes: #14183
Signed-off-by: zeertzjq <zeertzjq@outlook.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Not able to use diff() with 'diffexpr'
(rickhowe, after v9.1.0096)
Solution: Use a default context length of 0, update diff() help text,
add a test for using diff() with 'diffexpr'
(Yegappan Lakshmanan)
closes: #14013
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: diff() function uses 'diffexpr'
(rickhowe)
Solution: Make diff() always use internal diff(), add support for
unified diff context length, sort diff() options in help
(Yegappan Lakshmanan)
fixes: #13989closes: #14010
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Not able to build without FEAT_DIFF
(John Marriott, after 9.1.0071)
Solution: Adjust #ifdefs
(Yegappan Lakshmanan)
closes: #13964
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Need a diff() Vim script function
Solution: Add the diff() Vim script function using the
xdiff internal diff library, add support for
"unified" and "indices" mode.
(Yegappan Lakshmanan)
fixes: #4241closes: #12321
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: is*() and to*() function may be unsafe
Solution: Add SAFE_* macros and start using those instead
(Keith Thompson)
Use SAFE_() macros for is*() and to*() functions
The standard is*() and to*() functions declared in <ctype.h> have
undefined behavior for negative arguments other than EOF. If plain char
is signed, passing an unchecked value from argv for from user input
to one of these functions has undefined behavior.
Solution: Add SAFE_*() macros that cast the argument to unsigned char.
Most implementations behave sanely for negative arguments, and most
character values in practice are non-negative, but it's still best
to avoid undefined behavior.
The change from #13347 has been omitted, as this has already been
separately fixed in commit ac709e2fc0
(v9.0.2054)
fixes: #13332closes: #13347
Signed-off-by: Keith Thompson <Keith.S.Thompson@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: cannot complete option values
Solution: Add completion functions for several options
Add cmdline tab-completion for setting string options
Add tab-completion for setting string options on the cmdline using
`:set=` (along with `:set+=` and `:set-=`).
The existing tab completion for setting options currently only works
when nothing is typed yet, and it only fills in with the existing value,
e.g. when the user does `:set diffopt=<Tab>` it will be completed to
`set diffopt=internal,filler,closeoff` and nothing else. This isn't too
useful as a user usually wants auto-complete to suggest all the possible
values, such as 'iblank', or 'algorithm:patience'.
For set= and set+=, this adds a new optional callback function for each
option that can be invoked when doing completion. This allows for each
option to have control over how completion works. For example, in
'diffopt', it will suggest the default enumeration, but if `algorithm:`
is selected, it will further suggest different algorithm types like
'meyers' and 'patience'. When using set=, the existing option value will
be filled in as the first choice to preserve the existing behavior. When
using set+= this won't happen as it doesn't make sense.
For flag list options (e.g. 'mouse' and 'guioptions'), completion will
take into account existing typed values (and in the case of set+=, the
existing option value) to make sure it doesn't suggest duplicates.
For set-=, there is a new `ExpandSettingSubtract` function which will
handle flag list and comma-separated options smartly, by only suggesting
values that currently exist in the option.
Note that Vim has some existing code that adds special handling for
'filetype', 'syntax', and misc dir options like 'backupdir'. This change
preserves them as they already work, instead of converting to the new
callback API for each option.
closes: #13182
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: "clear" macros are not always used.
Solution: Use ALLOC_ONE, VIM_CLEAR, CLEAR_POINTER and CLEAR_FIELD in more
places. (Yegappan Lakshmanan, closes#12104)
Problem: FOR_ALL_ macros are defined in an unexpected file.
Solution: Move FOR_ALL_ macros to macros.h. Add FOR_ALL_HASHTAB_ITEMS.
(Yegappan Lakshmanan, closes#12109)
