[go: up one dir, main page]

Search results

149 packages found

Split text into tokens.

published 1.0.0 8 years ago
M
Q
P

Time a JavaScript tokenizer

published 1.1.0 7 years ago
M
Q
P

A streaming JSON tokenizer

published 1.1.0 7 years ago
M
Q
P
M
Q
P

A connector for the Trustami.AI API.

published 1.0.4 a year ago
M
Q
P

A gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs

published 1.1.0-laosbpatch.0 3 years ago
M
Q
P

Create a snapdragon token. Used by the snapdragon lexer, but can also be used by plugins.

published 4.0.0 6 years ago
M
Q
P

Lexical analyzer built in Javascript

published 1.0.4 6 years ago
M
Q
P

Tokenize Excel formulas

published 3.0.0 3 years ago
M
Q
P

Easily scan a string with an object of regex patterns to produce an array of tokens. ~100 sloc.

published 1.0.0 6 years ago
M
Q
P

tokenize sort string

published 1.0.4 6 years ago
M
Q
P

Fast (~300 MB/sec) and light (~1.3 kb) JSON/UTF-8 tokenizer for creating custom parsers

published 2.2.2 7 years ago
M
Q
P

Get search tokens from an INSA/NASA station name.

published 1.1.0 5 years ago
M
Q
P

Quick writer for transforming tokenized JSON back into JSON/UTF-8 output (works with qb-json-tok)

published 2.1.1 8 years ago
M
Q
P

Uses esprima to extract line and block comments from a string of JavaScript. Also optionally parses code context (the next line of code after a comment).

published 1.1.1 6 years ago
M
Q
P

Basic English tokenizer

published 1.0.6 8 years ago
M
Q
P

Binary wrapper for Sedlex_ppx - A lexer generator for OCaml, similar to ocamllex, but supporting Unicode.

published 20.0.0 5 years ago
M
Q
P

transpiles GLSL tokens from v100 to v300 es

published 1.2.0 9 years ago
M
Q
P
M
Q
P

A HTML tokenizer

published 1.0.2 8 years ago
M
Q
P