keywords:tokenizer - npm search

js-tokens

Tiny JavaScript tokenizer.

published version 9.0.1, 11 days ago5737 dependents licensed under $MIT

220,421,525

css-tree

A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

lahmatiy

published version 3.0.1, a month ago891 dependents licensed under $MIT

130,921,255

ret

Tokenizes a string that represents a regular expression.

fent

published version 0.5.0, 2 years ago823 dependents licensed under $MIT

80,895,424

micromark

small commonmark compliant markdown parser with positional info and concrete tokens

wooorm

published version 4.0.1, 21 days ago116 dependents licensed under $MIT

36,828,611

strtok3

A promise based streaming tokenizer

borewit

published version 10.0.1, 2 days ago52 dependents licensed under $MIT

20,244,066

@tokenizer/token

TypeScript definition for strtok3 token

borewit

published version 0.3.0, 3 years ago9 dependents licensed under $MIT

19,125,789

@csstools/css-tokenizer

Tokenize CSS

romainmenke

published version 3.0.3, a month ago41 dependents licensed under $MIT

16,763,726

stream-json is the micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API. I

elazutkin

published version 1.9.1, 21 days ago315 dependents licensed under $BSD-3-Clause

6,186,297

chevrotain

Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers

bd82

published version 11.0.3, a year ago321 dependents licensed under $Apache-2.0

5,922,500

scss-tokenizer

A tokenzier for Sass' SCSS syntax

xzyfer

published version 0.4.3, 2 years ago41 dependents licensed under $MIT

5,771,031

leac

Lexer / tokenizer

killy.mxi

published version 0.6.0, 2 years ago3 dependents licensed under $MIT

6,547,688

simple-html-tokenizer

Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.

rwjblue

published version 0.5.11, 4 years ago49 dependents licensed under $MIT

1,649,461

glsl-tokenizer

r/w stream of glsl tokens

dfcreative

published version 2.1.5, 6 years ago77 dependents licensed under $MIT

1,118,761

parenthesis

Parse parentheses from a string

dy

published version 3.1.8, 3 years ago62 dependents licensed under $MIT

1,042,719

natural

General natural language (tokenizing, stemming (English, Russian, Spanish), part-of-speech tagging, sentiment analysis, classification, inflection, phonetics, tfidf, WordNet, jaro-winkler, Levenshtein distance, Dice's Coefficient) facilities for node.