tinysearch is a lightweight, fast, full-text search engine. It is designed for static websites.
tinysearch is written in Rust, and then compiled to WebAssembly to run in a browser.
It can be used together with static site generators such as Jekyll,
Hugo, zola,
Cobalt, or Pelican.
tinysearch is a Rust/WASM port of the Python code from the article "Writing a full-text search engine using Bloom filters". It can be seen as an alternative to lunr.js and elasticlunr, which are too heavy for smaller websites and load a lot of JavaScript.
Under the hood it uses a Xor Filter -- a datastructure for fast approximation of set membership that is smaller than bloom and cuckoo filters. Each blog post gets converted into a filter that will then be serialized to a binary blob using bincode. Please not that the underlying technologies are subject to change.
The test index file of my blog with around 40 posts creates a WASM payload of 99kB (49kB gzipped, 40kB brotli).
That is smaller than the demo image above; so yes.
- Only searches for entire words. As a consequence there are no search suggestions (yet). This is a necessary tradeoff for reducing memory usage. A trie datastructure was about 10x bigger than the xor filters. New research on compact datastructures for prefix searches might lift this limitation in the future.
- Since we bundle all search indices for all articles into one static binary, we recommend to only use it for small- to medium-size websites. Expect around 4 kB uncompressed per article (~2 kb compressed).
wasm-pack is required to build the WASM module. Install it with
cargo install wasm-packTo optimize the JavaScript output, you'll also need terser:
npm install terser -g
If you want to make the WebAssembly as small as possible, we recommend to install binaryen as well. On macOS you can install it with homebrew:
brew install binaryenAlternatively, you can download the binary from the release page or use your OS package manager.
After that, you can install tinysearch itself:
cargo install tinysearch
As an input, we require a JSON index file, which contains the content to index. Here is an example file.
Once you created the index, you can run
tinysearch fixtures/index.json
ℹ️ You can take a look at the code examples for different static site generators here.
ℹ️ The body field in the JSON document is optional and can be skipped to just index post titles.
This will create a WASM module and the JavaScript glue code to integrate it into
your homepage. You can open the demo.html from any webserver to see the
result.
For example, Python has a built-in webserver that can be used for a quick test:
python3 -m http.server
then browse to http://0.0.0.0:8000/demo.html to see the result.
For advanced usage options, try
tinysearch --help
Please check what's required to host WebAssembly in production -- you will need to explicitly set gzip mime types.
If you don't have a full Rust setup, you can also use our nightly-built Docker images.
WASM_REPO: Overwrite the wasm-pack repositoryWASM_BRANCH: Overwrite the repository branch to useTINY_REPO: Overwrite repository of tinysearchTINY_BRANCH: Overwrite tinysearch branch
Here is to quickly try tinysearch with Docker:
# Download a sample blog index from endler.dev
curl -O https://raw.githubusercontent.com/tinysearch/tinysearch/master/fixtures/index.json
# Create the WASM output
docker run -v $PWD:/tmp tinysearch/cli index.jsonBy default, the most recent stable Alpine Rust image is used. To get nightly, run
docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .The following websites use tinysearch:
Are you using tinysearch, too? Add your site here!
- Matthias Endler (@mre)
- Jorge-Luis Betancourt (@jorgelbg)
- Mad Mike (@fluential)
tinysearch is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.