VectorCode Command-line Tool

• Installation
  • Install from Source
  • Migration from pipx
  • Chromadb
  • For Windows Users
  • Legacy Environments
  • Nix
• Getting Started
  • Refreshing Embeddings
  • If Anything Goes Wrong...
• Advanced Usage
  • Initialising a Project
    • Git Hooks
  • Configuring VectorCode
  • Vectorising Your Code
    • File Specs
  • Making a Query
  • Listing All Collections
  • Removing a Collection
  • Checking Project Setup
  • Cleaning up
  • Inspecting and Manupulating Files in an Indexed Project
  • Debugging and Diagnosing
    • Profiling
    • Post-mortem debugging
• Shell Completion
• Hardware Acceleration
• For Developers
  • Working with Results
    • vectorcode query
    • vectorcode vectorise
    • vectorcode ls
    • vectorcode files ls
  • LSP Mode
  • MCP Server
  • Writing Prompts

Installation

The CLI supports Python 3.11~3.13. You may also need a fairly recent c++/rust compiler because the core components of the vector database (ChromaDB) contains c++ and rust code.

The recommended way of installation is through uv, which will create a virtual environment for the package itself that doesn't mess up with your system Python or project-local virtual environments.

After installing uv, run:

uv tool install "vectorcode<1.0.0"

in your shell. To specify a particular version of Python, use the --python flag. For example, uv tool install vectorcode --python python3.11. For hardware accelerated embedding, refer to the relevant section. If you want a CPU-only installation without CUDA dependencies required by default by PyTorch, run:

uv tool install "vectorcode<1.0.0" --index https://download.pytorch.org/whl/cpu --index-strategy unsafe-best-match

If you need to install multiple dependency group (for LSP or MCP), you can use the following syntax:

uv tool install "vectorcode[lsp,mcp]<1.0.0"

Install from Source

To install from source, either git clone this repository and run uv tool install <path_to_vectorcode_repo>, or use pipx:

pipx install git+https://github.com/Davidyz/VectorCode

Migration from pipx

The motivation behind the change from pipx to uv tool is mainly the performance. The caching mechanism in uv makes it a lot faster than pipx for a lot of operations. If you installed VectorCode via pipx, you can continue to use pipx to manage your VectorCode installation. If you wish to switch to uv, you need to uninstall VectorCode using pipx and then use uv to install it as described above. All your VectorCode configurations and database files will work out of the box on your new install.

Chromadb

Chromadb is the vector database used by VectorCode to store and retrieve the code embeddings. Although it is already bundled with VectorCode and you can absolutely use VectorCode just fine, it is recommended to set up a standalone local server (they provides detailed instructions through docker and systemd), because this will significantly reduce the IO overhead and avoid potential race condition.

If you're setting up a standalone ChromaDB server, I recommend sticking to v0.6.3, because VectorCode is not ready for the upgrade to ChromaDB 1.0 yet.

For Windows Users

Windows support is not officially tested at this moment. This PR tracks my progress trying to provide better experiences for windows users.

Legacy Environments

If your environment doesn't support numpy version 2.0+, the default, unconstrained numpy may not work for you. In this case, you can try installing the package by uv tool install 'vectorcode[legacy]', which enforces numpy v1.x. If this doesn't help, please open an issue with your OS, CPU architecture, python version and the vectorcode virtual environment (uv tool run --from=vectorcode python -m ensurepip && uv tool run --from=vectorcode python -m pip freeze).

Nix

A community-maintained Nix package is available here. If you're using nix to install a standalone Chromadb server, make sure to stick to 0.6.3.

If you install via Nix and run into an issue, please try to reproduce with the PyPi package (install via uv or pipx). If it's not reproducible on the non-nix package, I may close the issue immediately.

Getting Started

cd into your project root repo, and run:

vectorcode init

This will initialise the project for VectorCode and create a .vectorcode directory in your project root. This is where you keep your configuration file for VectorCode, if any.

After that, you can start vectorising files for the project.

vectorcode vectorise src/**/*.py

VectorCode doesn't track file changes, so you need to re-vectorise edited files. You may automate this by a git pre-commit hook, etc. See the advanced usage section for examples to set them up.

Ideally, you should try to vectorise all source code in the repo, but for large repos you may experience slow queries. If that happens, try to vectorcode drop the project and only vectorise files that are important or informative.

And now, you're ready to make queries that will retrieve the relevant documents:

vectorcode query reranker -n 3

This will try to find the 3 most relevant documents in the embedding database that are related to the query reranker. You can pass multiple query words:

vectorcode query embedding reranking -n 3

or if you want to query a sentence, wrap them in quotation mark:

vectorcode query "How to configure reranker model"

If things are going right, you'll see some paths being printed, followed by their content. These are the selected documents that are relevant to the query.

If you want to wipe the embedding for the repository (to use a new embedding function or after an upgrade with breaking changes), use

vectorcode drop

To see a full list of CLI options and tricks to optimise the retrieval, keep reading or use the --help flag.

Refreshing Embeddings

To maintain the accuracy of the vector search, it's important to keep your embeddings up-to-date. You can simply run the vectorise subcommand on a file to refresh the embedding for that file. Apart from that, the CLI provides a vectorcode update subcommand, which updates the embeddings for all files that are currently indexed by VectorCode for the current project.

If you want something more automagic, check out the advanced usage section about setting up git hooks to trigger automatic embedding updates when you commit/checkout to a different tag.

If Anything Goes Wrong...

Please try the following and see if any of these fix your issue:

• drop the collection and re-index it, because there may be changes in the way embeddings are stored in the database;
• upgrade/re-install the CLI (via pipx or however you installed VectorCode).

Advanced Usage

Initialising a Project

For each project, VectorCode creates a collection (similar to tables in traditional databases) and puts the code embeddings in the corresponding collection. In the root directory of a project, you may run vectorcode init. This will initialise the repository with a subdirectory project_root/.vectorcode/. This will mark this directory as a project root, a concept that will later be used to construct the collection. You may put a config.json file in project_root/.vectorcode. This file may be used to store project-specific settings such as embedding functions and database entry point (more on this later). If you already have a global configuration file at ~/.config/vectorcode/config.json, it will be copied to project_root/.vectorcode/config.json when you run vectorcode init. When a project-local config file is present, the global configuration file is ignored to avoid confusion.

The same logics apply to file specs, which tells VectorCode what file it should (or shouldn't) vectorise. If you created a file spec ~/.config/vectorcode/vectorcode.include or ~/.config/vectorcode/vectorcode.exclude, they will be copied to the project-local config directory (project_root/.vectorcode). They also serve as the fallback value if no project-local specs are present.

If you skip vectorcode init, VectorCode will look for a directory that contains .git/ subdirectory and use it as the project root. In this case, the default global configuration will be used. If .git/ does not exist, VectorCode falls back to using the current working directory as the project root.

Git Hooks

To keep the embeddings up-to-date, you may find it useful to set up some git hooks. The init subcommand provides a --hooks flag which helps you manage hooks when working with a git repository. You can put some custom hooks in ~/.config/vectorcode/hooks/ and the vectorcode init --hooks command will pick them up and append them to your existing hooks, or create new hook scripts if they don't exist yet. The custom hook files should be named the same as they would be under the .git/hooks directory. For example, a pre-commit hook would be named ~/.config/vectorcode/hooks/pre-commit.

By default, there are 2 pre-defined hooks:

1. A pre-commit hook that vectorises the modified files.
2. A post-checkout hook that:
   • vectorises the full repository if it's an initial commit/clone and a vectorcode.include spec is available (either locally in the project or globally);
   • vectorises the files changed by the checkout.

Both hooks will only be triggered on repositories that have a .vectorcode directory in them.

Configuring VectorCode

Since 0.6.4, VectorCode adapted a json5 parser for loading configuration. VectorCode will now look for config.json5 in configuration directories, and if it doesn't find one, it'll look for config.json too. Regardless of the filename extension, the json5 syntax will be accepted. This allows you to leave trailing comma in the config file, as well as writing comments (//). This can be very useful if you're experimenting with the configs.

The JSON configuration file may hold the following values:

• embedding_function: string, one of the embedding functions supported by Chromadb (find more here and here). For example, Chromadb supports Ollama as chromadb.utils.embedding_functions.OllamaEmbeddingFunction, and the corresponding value for embedding_function would be OllamaEmbeddingFunction. Default: SentenceTransformerEmbeddingFunction;

• embedding_params: dictionary, stores whatever initialisation parameters your embedding function takes. For OllamaEmbeddingFunction, if you set embedding_params to:

  {
    "url": "http://127.0.0.1:11434/api/embeddings",
    "model_name": "nomic-embed-text"
  }

  Then the embedding function object will be initialised as OllamaEmbeddingFunction(url="http://127.0.0.1:11434/api/embeddings", model_name="nomic-embed-text"). Default: {};

• embedding_dims: integer or null, the number of dimensions to truncate the embeddings to. Make sure your model supports Matryoshka Representation Learning (MRL) before using this. Learn more about MRL here. When set to null (or unset), the embeddings won't be truncated;

• db_url: string, the url that points to the Chromadb server. VectorCode will start an HTTP server for Chromadb at a randomly picked free port on localhost if your configured http://host:port is not accessible. Default: http://127.0.0.1:8000;

• db_path: string, Path to local persistent database. If you didn't set up a standalone Chromadb server, this is where the files for your database will be stored. Default: ~/.local/share/vectorcode/chromadb/;

• db_log_path: string, path to the directory where the built-in chromadb server will write the log to. Default: ~/.local/share/vectorcode/;

• chunk_size: integer, the maximum number of characters per chunk. A larger value reduces the number of items in the database, and hence accelerates the search, but at the cost of potentially truncated data and lost information. Default: 2500. To disable chunking, set it to a negative number;

• overlap_ratio: float between 0 and 1, the ratio of overlapping/shared content between 2 adjacent chunks. A larger ratio improves the coherence of chunks, but at the cost of increasing number of entries in the database and hence slowing down the search. Default: 0.2. Starting from 0.4.11, VectorCode will use treesitter to parse languages that it can automatically detect. It uses pygments to guess the language from filename, and tree-sitter-language-pack to fetch the correct parser. overlap_ratio has no effects when treesitter works. If VectorCode fails to find an appropriate parser, it'll fallback to the legacy naive parser, in which case overlap_ratio works exactly in the same way as before;

• query_multiplier: integer, when you use the query command to retrieve n documents, VectorCode will check n * query_multiplier chunks and return at most n documents. A larger value of query_multiplier guarantees the return of n documents, but with the risk of including too many less-relevant chunks that may affect the document selection. Default: -1 (any negative value means selecting documents based on all indexed chunks);

• reranker: string, the reranking method to use. Currently supports NaiveReranker (sort chunks by the "distance" between the embedding vectors) and CrossEncoderReranker (using sentence-transformers cross-encoder ).

• reranker_params: dictionary, similar to embedding_params. The options passed to the reranker class constructor. For CrossEncoderReranker, these are the options passed to the CrossEncoder class. For example, if you want to use a non-default model, you can use the following:

  {
    "reranker_params": {
      "model_name_or_path": "your_model_here"
    }
  }

• db_settings: dictionary, works in a similar way to embedding_params, but for Chromadb client settings so that you can configure authentication for remote Chromadb;

• hnsw: a dictionary of hnsw settings that may improve the query performances or avoid runtime errors during queries. It's recommended to re-vectorise the collection after modifying these options, because some of the options can only be set during collection creation. Example (and default):

  "hnsw": {
    "hnsw:M": 64,
  }

• filetype_map: dict[str, list[str]], a dictionary where keys are language name and values are lists of Python regex patterns that will match file extensions. This allows overriding automatic language detection and specifying a treesitter parser for certain file types for which the language parser cannot be correctly identified (e.g., .phtml files containing both php and html). Example configuration:

  "filetype_map": {
    "php": ["^phtml$"]
  }

• chunk_filters: dict[str, list[str]], a dictionary where the keys are language name and values are lists of Python regex patterns that will match chunks to be excluded from being vectorised. This only applies to languages supported by treesitter chunker. By default, no filters will be added. Example configuration:

  "chunk_filters": {
    "python": ["^[^a-zA-Z0-9]+$"], // multiple patterns will be merged (unioned)
    // or you can use wildcard to match any languages that has no dedicated filters:
    "*": ["^[^a-zA-Z0-9]+$"],
  }

• encoding: string, alternative encoding used for this project. By default this project uses utf8. When this is set, VectorCode will decode files with the specified encoding, unless you choose to override this with the --encoding command line flag. You can also set this to _auto, which uses charset-normalizer to automatically detect the encoding, but this is not very accurate, especially on small files.

See the wiki for an example of the default configuration.

Vectorising Your Code

Run vectorcode vectorise <path_to_your_file> or vectorcode vectorise <directory> -r. There are a few extra tweaks you may use:

• chunk size: embedding APIs may truncate long documents so that the documents can be handled by the embedding models. To solve this, VectorCode implemented basic chunking features that chunks the documents into smaller segments so that the embeddings are more representative of the code content. To adjust the chunk size when vectorising, you may either set the chunk_size option in the JSON configuration file, or use --chunk_size/-c parameter of the vectorise command to specify the maximum number of characters per chunk;
• overlapping ratio: when the chunk size is set to