patch 8.2.0149: maintaining a Vim9 branch separately is more work
Problem: Maintaining a Vim9 branch separately is more work. Solution: Merge the Vim9 script changes.
This commit is contained in:
Bram Moolenaar
344
README_VIM9.md
Normal file
344
README_VIM9.md
Normal file
@ -0,0 +1,344 @@
|
||||
|
||||
|
||||
# What is Vim9?
|
||||
|
||||
This is an experimental side of [Vim](https://github.com/vim/vim).
|
||||
It explores ways of making Vim script faster and better.
|
||||
|
||||
WARNING: The Vim9 script features are in the early stages of development,
|
||||
anything can break!
|
||||
|
||||
# Why Vim9?
|
||||
|
||||
## 1. FASTER VIM SCRIPT
|
||||
|
||||
The third item on the poll results of 2018, after popup windows and text
|
||||
properties, is faster Vim script. So how do we do that?
|
||||
|
||||
I have been throwing some ideas around, and soon came to the conclusion
|
||||
that the current way functions are called and executed, with
|
||||
dictionaries for the arguments and local variables, is never going to be
|
||||
very fast. We're lucky if we can make it twice as fast. The overhead
|
||||
of a function call and executing every line is just too high.
|
||||
|
||||
So what then? We can only make something fast by having a new way of
|
||||
defining a function, with similar but different properties of the old
|
||||
way:
|
||||
* Arguments are only available by name, not through the a: dictionary or
|
||||
the a:000 list.
|
||||
* Local variables are not available in an l: dictionary.
|
||||
* A few more things that slow us down, such as exception handling details.
|
||||
|
||||
I Implemented a "proof of concept" and measured the time to run a simple
|
||||
for loop with an addition (Justin used this example in his presentation,
|
||||
full code is below):
|
||||
|
||||
``` vim
|
||||
let sum = 0
|
||||
for i in range(1, 2999999)
|
||||
let sum += i
|
||||
endfor
|
||||
```
|
||||
|
||||
| how | time in sec |
|
||||
| --------| -------- |
|
||||
| Vim old | 5.018541 |
|
||||
| Python | 0.369598 |
|
||||
| Lua | 0.078817 |
|
||||
| Vim new | 0.073595 |
|
||||
|
||||
That looks very promising! It's just one example, but it shows how much
|
||||
we can gain, and also that Vim script can be faster than builtin
|
||||
interfaces.
|
||||
|
||||
In practice the script would not do something useless as counting but change
|
||||
the text. For example, re-indent all the lines:
|
||||
|
||||
``` vim
|
||||
let totallen = 0
|
||||
for i in range(1, 100000)
|
||||
call setline(i, ' ' .. getline(i))
|
||||
let totallen += len(getline(i))
|
||||
endfor
|
||||
```
|
||||
|
||||
| how | time in sec |
|
||||
| --------| -------- |
|
||||
| Vim old | 0.853752 |
|
||||
| Python | 0.304584 |
|
||||
| Lua | 0.286573 |
|
||||
| Vim new | 0.190276 |
|
||||
|
||||
The differences are smaller, but Vim 9 script is clearly the fastest.
|
||||
|
||||
How does Vim9 script work? The function is first compiled into a sequence of
|
||||
instructions. Each instruction has one or two parameters and a stack is
|
||||
used to store intermediate results. Local variables are also on the
|
||||
stack, space is reserved during compilation. This is a fairly normal
|
||||
way of compilation into an intermediate format, specialized for Vim,
|
||||
e.g. each stack item is a typeval_T. And one of the instructions is
|
||||
"execute Ex command", for commands that are not compiled.
|
||||
|
||||
|
||||
## 2. PHASING OUT INTERFACES
|
||||
|
||||
Attempts have been made to implement functionality with built-in script
|
||||
languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much
|
||||
foothold, for various reasons.
|
||||
|
||||
Instead of using script language support in Vim:
|
||||
* Encourage implementing external tools in any language and communicate
|
||||
with them. The job and channel support already makes this possible.
|
||||
Really any language can be used, also Java and Go, which are not
|
||||
available built-in.
|
||||
* Phase out the built-in language interfaces, make maintenance a bit easier
|
||||
and executables easier to build. They will be kept for backwards
|
||||
compatibility, no new features.
|
||||
* Improve the Vim script language, it is used to communicate with the external
|
||||
tool and implements the Vim side of the interface. Also, it can be used when
|
||||
an external tool is undesired.
|
||||
|
||||
All together this creates a clear situation: Vim with the +eval feature
|
||||
will be sufficient for most plugins, while some plugins require
|
||||
installing a tool that can be written in any language. No confusion
|
||||
about having Vim but the plugin not working because some specific
|
||||
language is missing. This is a good long term goal.
|
||||
|
||||
Rationale: Why is it better to run a tool separately from Vim than using a
|
||||
built-in interface and interpreter? Take for example something that is
|
||||
written in Python:
|
||||
* The built-in interface uses the embedded python interpreter. This is less
|
||||
well maintained than the python command. Building Vim with it requires
|
||||
installing developer packages. If loaded dynamically there can be a version
|
||||
mismatch.
|
||||
* When running the tool externally the standard python command can be used,
|
||||
which is quite often available by default or can be easily installed.
|
||||
* The built-in interface has an API that is unique for Vim with Python. This is
|
||||
an extra API to learn.
|
||||
* A .py file can be compiled into a .pyc file and execute much faster.
|
||||
* Inside Vim multi-threading can cause problems, since the Vim core is single
|
||||
threaded. In an external tool there are no such problems.
|
||||
* The Vim part is written in .vim files, the Python part is in .py files, this
|
||||
is nicely separated.
|
||||
* Disadvantage: An interface needs to be made between Vim and Python.
|
||||
JSON is available for this, and it's fairly easy to use. But it still
|
||||
requires implementing asynchronous communication.
|
||||
|
||||
|
||||
## 3. BETTER VIM SCRIPT
|
||||
|
||||
To make Vim faster a new way of defining a function needs to be added.
|
||||
While we are doing that, since the lines in this function won't be fully
|
||||
backwards compatible anyway, we can also make Vim script easier to use.
|
||||
In other words: "less weird". Making it work more like modern
|
||||
programming languages will help. No surprises.
|
||||
|
||||
A good example is how in a function the arguments are prefixed with
|
||||
"a:". No other language I know does that, so let's drop it.
|
||||
|
||||
Taking this one step further is also dropping "s:" for script-local variables;
|
||||
everything at the script level is script-local by default. Since this is not
|
||||
backwards compatible it requires a new script style: Vim9 script!
|
||||
|
||||
It should be possible to convert code from other languages to Vim
|
||||
script. We can add functionality to make this easier. This still needs
|
||||
to be discussed, but we can consider adding type checking and a simple
|
||||
form of classes. If you look at JavaScript for example, it has gone
|
||||
through these stages over time, adding real class support and now
|
||||
TypeScript adds type checking. But we'll have to see how much of that
|
||||
we actually want to include in Vim script. Ideally a conversion tool
|
||||
can take Python, JavaScript or TypeScript code and convert it to Vim
|
||||
script, with only some things that cannot be converted.
|
||||
|
||||
Vim script won't work the same as any specific language, but we can use
|
||||
mechanisms that are commonly known, ideally with the same syntax. One
|
||||
thing I have been thinking of is assignments without ":let". I often
|
||||
make that mistake (after writing JavaScript especially). I think it is
|
||||
possible, if we make local variables shadow commands. That should be OK,
|
||||
if you shadow a command you want to use, just rename the variable.
|
||||
Using "let" and "const" to declare a variable, like in JavaScript and
|
||||
TypeScript, can work:
|
||||
|
||||
|
||||
``` vim
|
||||
def MyFunction(arg: number): number
|
||||
let local = 1
|
||||
let todo = arg
|
||||
const ADD = 88
|
||||
while todo > 0
|
||||
local += ADD
|
||||
--todo
|
||||
endwhile
|
||||
return local
|
||||
enddef
|
||||
```
|
||||
|
||||
The similarity with JavaScript/TypeScript can also be used for dependencies
|
||||
between files. Vim currently uses the `:source` command, which has several
|
||||
disadvantages:
|
||||
* In the sourced script, is not clear what it provides. By default all
|
||||
functions are global and can be used elsewhere.
|
||||
* In a script that sources other scripts, it is not clear what function comes
|
||||
from what sourced script. Finding the implementation is a hassle.
|
||||
* Prevention of loading the whole script twice must be manually implemented.
|
||||
|
||||
We can use the `:import` and `:export` commands from the JavaScript standard to
|
||||
make this much better. For example, in script "myfunction.vim" define a
|
||||
function and export it:
|
||||
|
||||
``` vim
|
||||
vim9script " Vim9 script syntax used here
|
||||
|
||||
let local = 'local variable is not exported, script-local'
|
||||
|
||||
export def MyFunction() " exported function
|
||||
...
|
||||
|
||||
def LocalFunction() " not exported, script-local
|
||||
...
|
||||
```
|
||||
|
||||
And in another script import the function:
|
||||
|
||||
``` vim
|
||||
vim9script " Vim9 script syntax used here
|
||||
|
||||
import MyFunction from 'myfunction.vim'
|
||||
```
|
||||
|
||||
This looks like JavaScript/TypeScript, thus many users will understand the
|
||||
syntax.
|
||||
|
||||
These are ideas, this will take time to design, discuss and implement.
|
||||
Eventually this will lead to Vim 9!
|
||||
|
||||
|
||||
## Code for sum time measurements
|
||||
|
||||
Vim was build with -O2.
|
||||
|
||||
``` vim
|
||||
func VimOld()
|
||||
let sum = 0
|
||||
for i in range(1, 2999999)
|
||||
let sum += i
|
||||
endfor
|
||||
return sum
|
||||
endfunc
|
||||
|
||||
func Python()
|
||||
py3 << END
|
||||
sum = 0
|
||||
for i in range(1, 3000000):
|
||||
sum += i
|
||||
END
|
||||
return py3eval('sum')
|
||||
endfunc
|
||||
|
||||
func Lua()
|
||||
lua << END
|
||||
sum = 0
|
||||
for i = 1, 2999999 do
|
||||
sum = sum + i
|
||||
end
|
||||
END
|
||||
return luaeval('sum')
|
||||
endfunc
|
||||
|
||||
def VimNew()
|
||||
let sum = 0
|
||||
for i in range(1, 2999999)
|
||||
let sum += i
|
||||
endfor
|
||||
return sum
|
||||
enddef
|
||||
|
||||
let start = reltime()
|
||||
echo VimOld()
|
||||
echo 'Vim old: ' .. reltimestr(reltime(start))
|
||||
|
||||
let start = reltime()
|
||||
echo Python()
|
||||
echo 'Python: ' .. reltimestr(reltime(start))
|
||||
|
||||
let start = reltime()
|
||||
echo Lua()
|
||||
echo 'Lua: ' .. reltimestr(reltime(start))
|
||||
|
||||
let start = reltime()
|
||||
echo VimNew()
|
||||
echo 'Vim new: ' .. reltimestr(reltime(start))
|
||||
```
|
||||
|
||||
## Code for indent time measurements
|
||||
|
||||
``` vim
|
||||
def VimNew(): number
|
||||
let totallen = 0
|
||||
for i in range(1, 100000)
|
||||
setline(i, ' ' .. getline(i))
|
||||
totallen += len(getline(i))
|
||||
endfor
|
||||
return totallen
|
||||
enddef
|
||||
|
||||
func VimOld()
|
||||
let totallen = 0
|
||||
for i in range(1, 100000)
|
||||
call setline(i, ' ' .. getline(i))
|
||||
let totallen += len(getline(i))
|
||||
endfor
|
||||
return totallen
|
||||
endfunc
|
||||
|
||||
func Lua()
|
||||
lua << END
|
||||
b = vim.buffer()
|
||||
totallen = 0
|
||||
for i = 1, 100000 do
|
||||
b[i] = " " .. b[i]
|
||||
totallen = totallen + string.len(b[i])
|
||||
end
|
||||
END
|
||||
return luaeval('totallen')
|
||||
endfunc
|
||||
|
||||
func Python()
|
||||
py3 << END
|
||||
cb = vim.current.buffer
|
||||
totallen = 0
|
||||
for i in range(0, 100000):
|
||||
cb[i] = ' ' + cb[i]
|
||||
totallen += len(cb[i])
|
||||
END
|
||||
return py3eval('totallen')
|
||||
endfunc
|
||||
|
||||
new
|
||||
call setline(1, range(100000))
|
||||
let start = reltime()
|
||||
echo VimOld()
|
||||
echo 'Vim old: ' .. reltimestr(reltime(start))
|
||||
bwipe!
|
||||
|
||||
new
|
||||
call setline(1, range(100000))
|
||||
let start = reltime()
|
||||
echo Python()
|
||||
echo 'Python: ' .. reltimestr(reltime(start))
|
||||
bwipe!
|
||||
|
||||
new
|
||||
call setline(1, range(100000))
|
||||
let start = reltime()
|
||||
echo Lua()
|
||||
echo 'Lua: ' .. reltimestr(reltime(start))
|
||||
bwipe!
|
||||
|
||||
new
|
||||
call setline(1, range(100000))
|
||||
let start = reltime()
|
||||
echo VimNew()
|
||||
echo 'Vim new: ' .. reltimestr(reltime(start))
|
||||
bwipe!
|
||||
```
|
||||
Reference in New Issue
Block a user