patch 9.1.1476: no easy way to deduplicate text

Problem:  no easy way to deduplicate text
Solution: add the :uniq ex command
          (Hirohito Higashi)

closes: #17538

Signed-off-by: Hirohito Higashi <h.east.727@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
This commit is contained in:
Hirohito Higashi
2025-06-23 21:42:36 +02:00
committed by Christian Brabandt
parent 159d392427
commit 74f0a77bb9
12 changed files with 929 additions and 14 deletions

View File

@ -1,4 +1,4 @@
*builtin.txt* For Vim version 9.1. Last change: 2025 Jun 07
*builtin.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@ -12143,6 +12143,7 @@ uniq({list} [, {func} [, {dict}]]) *uniq()* *E882*
:let newlist = uniq(copy(mylist))
< The default compare function uses the string representation of
each item. For the use of {func} and {dict} see |sort()|.
For deduplicating text in the current buffer see |:uniq|.
Returns zero if {list} is not a |List|.

View File

@ -1,4 +1,4 @@
*change.txt* For Vim version 9.1. Last change: 2025 May 28
*change.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@ -20,6 +20,7 @@ commands with the "." command.
5. Copying and moving text |copy-move|
6. Formatting text |formatting|
7. Sorting text |sorting|
8. Deduplicating text |deduplicating|
For inserting text see |insert.txt|.
@ -1895,6 +1896,7 @@ And a few warnings:
Vim has a sorting function and a sorting command. The sorting function can be
found here: |sort()|, |uniq()|.
Also see |:uniq|.
*:sor* *:sort*
:[range]sor[t][!] [b][f][i][l][n][o][r][u][x] [/{pattern}/]
@ -1904,7 +1906,7 @@ found here: |sort()|, |uniq()|.
With [!] the order is reversed.
With [i] case is ignored.
*:sort-l*
With [l] sort uses the current collation locale.
Implementation details: strcoll() is used to compare
strings. See |:language| to check or set the collation
@ -1937,13 +1939,14 @@ found here: |sort()|, |uniq()|.
With [b] sorting is done on the first binary number in
the line (after or inside a {pattern} match).
*:sort-u* *:sort-uniq*
With [u] (u stands for unique) only keep the first of
a sequence of identical lines (ignoring case when [i]
is used). Without this flag, a sequence of identical
lines will be kept in their original order.
Note that leading and trailing white space may cause
lines to be different.
When you just want to make things unique, use |:uniq|.
When /{pattern}/ is specified and there is no [r] flag
the text matched with {pattern} is skipped, so that
@ -1990,4 +1993,56 @@ The sorting can be interrupted, but if you interrupt it too late in the
process you may end up with duplicated lines. This also depends on the system
library function used.
==============================================================================
8. Deduplicating text *deduplicating* *unique*
Vim has a deduplicating function and a deduplicating command. The
deduplicating function can be found here: |uniq()|.
Also see |:sort-uniq|.
*:uni* *:uniq*
:[range]uni[q][!] [i][l][r][u] [/{pattern}/]
Remove duplicate lines that are adjacent to each other
in [range]. When no range is given, all lines are
processed.
With [i] case is ignored when comparing lines.
With [l] comparison uses the current collation locale.
See |:sort-l| for more details.
With [r] comparison is done on the text that matches
/{pattern}/ instead of the full line.
When /{pattern}/ is specified and [r] is not used, the
text matched with {pattern} is skipped and comparison
is done on what comes after the match.
'ignorecase' applies to the pattern, but 'smartcase'
is not used.
Instead of the slash any non-letter can be used.
For example, to remove adjacent duplicate lines based
on the second comma-separated field: >
:uniq r /[^,]*,/
< Or to keep only unique lines ignoring the first 5
characters: >
:uniq u /.\{5}/
< If {pattern} is empty (e.g. // is used), the last
search pattern is used.
With [u] only keep lines that do not repeat (i.e., are
not immediately followed by the same line).
With [!] only keep lines that are immediately followed
by a duplicate.
If both [!] and [u] are given, [u] is ignored and [!]
takes effect.
Note that leading and trailing white space, and lines
that are not adjacent, are not considered duplicates.
To remove all duplicates regardless of position, use
|:sort-u| or external tools.
vim:tw=78:ts=8:noet:ft=help:norl:

View File

@ -1,4 +1,4 @@
*index.txt* For Vim version 9.1. Last change: 2025 Jun 02
*index.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@ -1740,6 +1740,7 @@ tag command action ~
|:unabbreviate| :una[bbreviate] remove abbreviation
|:unhide| :unh[ide] open a window for each loaded file in the
buffer list
|:uniq| :uni[q] uniq lines
|:unlet| :unl[et] delete variable
|:unlockvar| :unlo[ckvar] unlock variables
|:unmap| :unm[ap] remove mapping

View File

@ -3331,6 +3331,9 @@ $quote eval.txt /*$quote*
:so repeat.txt /*:so*
:sor change.txt /*:sor*
:sort change.txt /*:sort*
:sort-l change.txt /*:sort-l*
:sort-u change.txt /*:sort-u*
:sort-uniq change.txt /*:sort-uniq*
:source repeat.txt /*:source*
:source! repeat.txt /*:source!*
:source-range repeat.txt /*:source-range*
@ -3565,6 +3568,8 @@ $quote eval.txt /*$quote*
:undolist undo.txt /*:undolist*
:unh windows.txt /*:unh*
:unhide windows.txt /*:unhide*
:uni change.txt /*:uni*
:uniq change.txt /*:uniq*
:unl eval.txt /*:unl*
:unlet eval.txt /*:unlet*
:unlet-$ eval.txt /*:unlet-$*
@ -6864,6 +6869,7 @@ debugger-support debugger.txt /*debugger-support*
debugger.txt debugger.txt /*debugger.txt*
dec-mouse options.txt /*dec-mouse*
decada_members ft_ada.txt /*decada_members*
deduplicating change.txt /*deduplicating*
deepcopy() builtin.txt /*deepcopy()*
default-constructor vim9class.txt /*default-constructor*
defaults.vim starting.txt /*defaults.vim*
@ -11015,6 +11021,7 @@ undofile() builtin.txt /*undofile()*
undotree() builtin.txt /*undotree()*
unicode mbyte.txt /*unicode*
uniq() builtin.txt /*uniq()*
unique change.txt /*unique*
unix os_unix.txt /*unix*
unlisted-buffer windows.txt /*unlisted-buffer*
up-down-motions motion.txt /*up-down-motions*

View File

@ -1,4 +1,4 @@
*version9.txt* For Vim version 9.1. Last change: 2025 Jun 16
*version9.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@ -41756,6 +41756,7 @@ Ex-Commands: ~
|:pbuffer| Edit buffer [N] from the buffer list in the preview
window
|:redrawtabpanel| Force updating the 'tabpanel'.
|:uniq| Deduplicate text in the current buffer.
Options: ~