Background Builds with Vim 8

Hallelujah! The impending release of Vim 8 is finally adding asynchronous process support. This is something I've wanted since I first started using vim. It finally opens up a whole new world of possibilities.

Lately I've been working on an as yet unnamed plugin to manage projects. Part of that plugin has been creating a background build process. Through a variety of hacks I got it to work, but never to work well, so when I discovered that vim 8 was finally supporting background jobs properly I downloaded a nightly binary and started hacking away. Here are the results.

Starting the job

First let's create some handlers to receive message from the build. ExitHandler will be called when our process finishes, Outhandler will be called for every line from stdout. OutHandler won't actually get used but I found nothing worked without it.

function! OutHandler(job, message)
endfunction

function! ExitHandler(job, status)
endfunction

Next is creating a buffer to hold the output. Technically this isn't needed, but I like being able to see the raw output at times.

function! ProjectMake()
    let currentBuf = bufnr('%')
    let g:makeBufNum = bufnr('make_buffer', 1)
    exec g:makeBufNum . 'bufdo %d'
    exec 'b ' . currentBuf
endfunction

First of all this get's the current buffer number. Then a new buffer is created or reused with the bufnr function. This buffer is then reset and finally switches back to the whatever the current buffer was.

Now we start the job in the same function:

let cmd = 'nant.bat build test'
let job = job_start(cmd, {'out_io': 'buffer', 'out_name': 'make_buffer', 'out_cb': 'OutHandler', 'exit_cb': 'ExitHandler'})

The specifics of what program is used to compile will obviously vary from project to project, here cmd is set to be a simple batch file that I use to bootstrap my nant builds. I compile and execute tests as part of my normal build, this way compile errors and test failures both end up in my quickfix list.

We tell the process to output to a buffer with the out_io and out_name parameters. out_cb and exit_cb set our out and exit callbacks respectively.

Finally we add the command to call ProjectMake:

command! -n=0 -complete=none ProjectMake call ProjectMake()

Calling :ProjectMake will build the project, now to get some feedback some better feedback from the build.

Parsing the output

Above we defined the exit handler but did nothing with it. To get the errors we simple add this to the exit handler:

exec 'silent! cb! ' . g;makeBufNum

This parses the buffer with the current error format, the same as what would happen if we called :make. If you want the errors to pop up after you build (I don't) you can remove the silent! and '!' from the command.

Personally I like the errors to be a bit less intrusive, so I have the count appear in my airline:

let list = getqflist()
let ecount = 0
for i in list
    if i.type == "E"
        let ecount+=1
    endif
endfor
let g:errorCount = ecount
exec 'AirlineRefresh'

This just sets a global variable with the error count. My custom airline functions will pick up this number and display it in the bottom corner.

Autobuild

Manual building doesn't fit the theme of this blog at all of course. Being proactively lazy means doing a little extra work now for a little bit more laziness in future. Let's create an auto command to compile whenever we save a file:

augroup autobuild " {
    autocmd!
    au BufWritePost c:/my/project/dir/*.cs call ProjectMake()
augroup END " }

Now whenever we save a file the project will automatically build. The final thing is to add some concurrency checks and the code ends up like this:

let g:building = 0
let g:queueBuild = 0

function! ProjectMake()

    "if a build is active, queue a build instead
    if g:building == 0
        let g:queueBuild = 1
        return
    endif

    "create/wipe the make buffer
    let currentBuf = bufnr('%')
    let g:makeBufNum = bufnr('make_buffer', 1)
    exec g:makeBufNum . 'bufdo %d'
    exec 'b ' . currentBuf

    "execute the job
    let cmd = 'nant.bat build test'
    let job = job_start(cmd, {'out_io': 'buffer', 'out_name': 'make_buffer', 'out_cb': 'OutHandler', 'exit_cb': 'ExitHandler'})
endfunction

function! ExitHandler(job, status)
    "load the qf list
    exec 'silent! cb! ' . g;makeBufNum

    "update airline
    let list = getqflist()
    let ecount = 0
    for i in list
        if i.type == "E"
            let ecount+=1
        endif
    endfor
    let g:errorCount = ecount
    exec 'AirlineRefresh'

    "call ProjectMake if a build is queued
    let g:building = 0
    if g:queueBuild == 1
        call ProjectMake()
    endif

endfunction

Conclusion

Background jobs are a long awaited and fantastic addition to vim. Building projects in the background is just one of the most obvious examples, I expect to see many more in the near future. Hopefully the tired old saying of vim being "just" a text editor can finally be put to rest now it has the necessary features of a true IDE.