Neopyter

The bridge between Neovim and Jupyter Lab, edit in Neovim and preview/run in Jupyter Lab.

• Neopyter
  • How does it work?
  • Screenshots
  • Requirements
  • Installation
    • JupyterLab Extension
    • Neovim plugin
  • Quick Start
  • Available Vim Commands
  • Integration
    • neoconf.nvim
    • nvim-cmp
    • nvim-treesitter-textobjects
  • API
  • Features
  • Acknowledges

How does it work?

This project includes two parts: a JupyterLab extension and a Neovim plugin

• The JupyterLab extension exposes functions of Jupyter lab, and provides a remote procedure call(RPC) service
• The Neovim plugin calls the RPC service when it receives events from Neovim via autocmd

This project provides two work modes for different network environments. If the browser where your jupyter lab is located cannot directly access nvim, you must use proxy mode; If you need to collaborate and use the same Jupyter with others, you must use direct mode

	direct	proxy
Architecture
Advantage	Lower communication costs Shareable JupyterLab instance	Lower Neovim load
Disadvantage	Higher Neovim load	Exclusive JupyterLab instance

• direct mode: (default, recommended) In this mode, neovim is server and neovim plugin(neopyter) is listening to remote_address, the browser where jupyter lab is located will connect to neovim

• proxy mode: In this mode, Jupyter lab server(server side, the host you run jupyter lab to start JupyterLab) is server and jupyter lab server extension(neopyter) is listening to ${IP}:{Port}, the neovim plugin(neopyter) will connect to ${IP}:{Port}

Ultimately, Neopyter can control Juppyter lab. Neopyter can implement abilities like jupynium.nvim.

Screenshots

	Completion	Cell Magic	Line Magic

Requirements

• 📔JupyterLab >= 4.0.0
• ✌️ Neovim nightly
  • 👍nvim-lua/plenary.nvim
  • 🤏AbaoFromCUG/websocket.nvim (optional for mode="direct")

Installation

JupyterLab Extension

To install the jupyterlab extension, execute:

pip install neopyter

Configure JupyterLab in side panel

• mode: Refer to the previous introduction about mode
• IP: If mode=proxy, set to the IP of the host where jupyter server is located. If proxy=direct, set to the IP of the host where neovim is located
• Port: Idle port of the IP's' host

NOTICE: all settings is saved to localStorage

Neovim plugin

With 💤lazy.nvim:

{
    "SUSTech-data/neopyter",
    ---@type neopyter.Option
    opts = {
        mode="direct",
        remote_address = "127.0.0.1:9001",
        file_pattern = { "*.ju.*" },
        on_attach = function(bufnr)
            -- do some buffer keymap
        end,
        highlight = {
            enable = true,
            shortsighted = false,
        },
        parser = {
            -- trim leading/tailing whitespace of cell
            trim_whitespace = false,
        }
    },
}

Suggest keymaps(neopyter don't provide default keymap):

on_attach = function(buf)
    local function map(mode, lhs, rhs, desc)
        vim.keymap.set(mode, lhs, rhs, { desc = desc, buffer = buf })
    end
    -- same, recommend the former
    map("n", "<C-Enter>", "<cmd>Neopyter execute notebook:run-cell<cr>", "run selected")
    -- map("n", "<C-Enter>", "<cmd>Neopyter run current<cr>", "run selected")

    -- same, recommend the former
    map("n", "<space>X", "<cmd>Neopyter execute notebook:run-all-above<cr>", "run all above cell")
    -- map("n", "<space>X", "<cmd>Neopyter run allAbove<cr>", "run all above cell")

    -- same, recommend the former, but the latter is silent
    map("n", "<space>nt", "<cmd>Neopyter execute kernelmenu:restart<cr>", "restart kernel")
    -- map("n", "<space>nt", "<cmd>Neopyter kernel restart<cr>", "restart kernel")

    map("n", "<S-Enter>", "<cmd>Neopyter execute runmenu:run<cr>", "run selected and select next")
    map("n", "<M-Enter>", "<cmd>Neopyter execute run-cell-and-insert-below<cr>", "run selected and insert below")

    map("n", "<F5>", "<cmd>Neopyter execute notebook:restart-run-all<cr>", "restart kernel and run all")
end

Quick Start

• Open JupyterLab jupyter lab, there is a sidebar named Neopyter, which display neopyter ip+port
• Open a *.ju.py file in neovim
• Now you can type # %% in Neovim to create a code cell.
  • You'll see everything you type below that will be synchronised in the browser

Available Vim Commands

• Status
  • :Neopyter status alias to :checkhealth neopyter currently
• Server
  • :Neopyter connect [remote 'ip:port'], e.g. :Neopyter command 127.0.0.1:9001, connect Jupyter lab manually
  • :Neopyter disconnect
• Sync
  • :Neopyter sync current, make sync current *.ju.* file with the currently open *.ipynb
  • :Neopyter sync [filename], e.g. :Neopyter sync main.ipynb
• Run
  • :Neopyter run current, same as Run>Run Selected Cell and Do not Advance menu in Jupyter lab
  • :Neopyter run allAbove, same as Run>Run All Above Selected Cell menu in Jupyter lab
  • :Neopyter run allBelow, same as Run>Run Selected Cell and All Below menu in Jupyter lab
  • :Neopyter run all, same as Run>Run All Cells menu in Jupyter lab
• Kernel
  • :Neopyter kernel restart, same as Kernel>Restart Kernel menu in Jupyter lab
  • :Neopyter kernel restartRunAll, same as Kernel>Restart Kernel and Run All Cells menu in Jupyter lab
• Jupyter
  • :Neopyter execute [command_id] [args], execute Jupyter lab's command directly, e.g. :Neopyter execute notebook:export-to-format {"format":"html"}

Integration

neoconf.nvim

If neoconf.nvim is available, neopyter will automatically register/read neoconf settings

.neoconf.json

{
  "neopyter": {
    "mode": "proxy",
    "remote_address": "127.0.0.1:9001"
  }
}

nvim-cmp

• nvim-cmp
• lspkind.nvim

local lspkind = require("lspkind")
local cmp = require("cmp")

cmp.setup({
    sources = cmp.config.sources({
        -- default: all source, maybe some noice
        { name = "neopyter" },
        -- only kernel source, like jupynium, support jupyterlab completer id:
        -- * "CompletionProvider:kernel"
        -- * "CompletionProvider:context"
        -- * "lsp" if jupyterlab-lsp is installed
        -- * ...
        -- { name = "neopyter", option={ completers = { "CompletionProvider:kernel" } } },
    }),
    formatting = {
        format = lspkind.cmp_format({
            mode = "symbol_text",
            maxwidth = 50,
            ellipsis_char = "...",
            menu = {
                neopyter = "[Neopyter]",
            },
            symbol_map = {
                -- specific complete item kind icon
                ["Magic"] = "🪄",
                ["Path"] = "📁",
                ["Dict key"] = "🔑",
                ["Instance"]="󱃻",
                ["Statement"]="󱇯",
            },
        }),
    },
)}

    -- menu item highlight
vim.api.nvim_set_hl(0, "CmpItemKindMagic", { bg = "NONE", fg = "#D4D434" })
vim.api.nvim_set_hl(0, "CmpItemKindPath", { link = "CmpItemKindFolder" })
vim.api.nvim_set_hl(0, "CmpItemKindDictkey", { link = "CmpItemKindKeyword" })
vim.api.nvim_set_hl(0, "CmpItemKindInstance", { link = "CmpItemKindVariable" })
vim.api.nvim_set_hl(0, "CmpItemKindStatement", { link = "CmpItemKindVariable" })

More information, see nvim-cmp wiki

nvim-treesitter-textobjects

Supported captures in textobjects query group

• @cell
  • @cell.code
  • @cell.magic
  • @cell.markdown
  • @cell.raw
  • @cell.special
• @cellseparator
  • @cellseparator.code
  • @cellseparator.magic
  • @cellseparator.markdown
  • @cellseparator.raw
  • @cellseparator.special
• @cellbody
  • @cellbody.code
  • @cellbody.magic
  • @cellbody.markdown
  • @cellbody.raw
  • @cellbody.special
• @cellcontent
  • @cellcontent.code
  • @cellcontent.magic
  • @cellcontent.markdown
  • @cellcontent.raw
  • @cellcontent.special
• @cellborder
  • @cellborder.start
    • @cellborder.start.markdown
    • @cellborder.start.raw
    • @cellborder.start.special
  • @cellborder.end
    • @cellborder.end.markdown
    • @cellborder.end.raw
    • @cellborder.end.special
• @linemagic

require'nvim-treesitter.configs'.setup {
    textobjects = {
        move = {
            enable = true,
            goto_next_start = {
                ["]j"] = "@cellseparator",
                ["]c"] = "@cellcontent",
            },
            goto_previous_start = {
                ["[j"] = "@cellseparator",
                ["[c"] = "@cellcontent",
            },
        },
    },
}

API

Neopyter provides rich lua APIs

• Jupyter Lab

  • Neopyter execute ... <-> require("neopyter.jupyter").jupyterlab:execute_command(...)
  • All APIs see :lua =require("neopyter.jupyter.jupyterlab").__injected_methods

• Notebook

  • :Neopyter run current <-> require("neopyter.jupyter").notebook:run_selected_cell()
  • :Neopyter run allAbove <-> require("neopyter.jupyter").notebook:run_all_above()
  • :Neopyter run allBelow <-> require("neopyter.jupyter").notebook:run_all_below()
  • All APIs see :lua =require("neopyter.jupyter.notebook").__injected_methods

Features

• Neovim
  • Full sync
  • Partial sync
  • Scroll view automatically
  • Activate cell automatically
  • Save notebook automatically
  • Completion
    • Magic completion item
    • Path completion item
    • Disable others?
  • Tree-sitter
    • Highlight
      • Separator+non-code
      • Shortsighted
    • Textobjects
    • Fold
  • Kernel manage
    • Restart kernel
    • Restart kernel and run all
  • Run cell
    • Run selected cell
    • Run all above selected cell
    • Run selected cell and all below
    • Run all cell
  • Sync
    • Set synchronized .ipynb manually
  • Notebook manager
    • Open corresponding notebook if exists
    • Sync with untitled notebook default
    • Close notebook when buffer unload
• Jupyter Lab
  • Settings
    • TCP server host/port settings
  • Status Sidebar
    • Settings ip:port
    • Display client info
• Performance
  • Rewrite RpcClient, support async RPC request vim.rpcrequest and vim.rpcnotify
• Document
  • API Document

Acknowledges

• jupynium.nvim: Selenium-automated Jupyter Notebook that is synchronised with Neovim in real-time.