basant307/AI_Governance_Project
048
1# peberminta2 345[](https://codecov.io/gh/mxxii/peberminta)6[](https://github.com/mxxii/peberminta/blob/main/LICENSE)7[](https://www.npmjs.com/package/peberminta)8[](https://www.npmjs.com/package/peberminta)9[](https://deno.land/x/peberminta)10 11Simple, transparent parser combinators toolkit that supports tokens of any type.12 13For when you wanna do weird things with parsers.14 15 16## Features17 18- **Well typed** - written in TypeScript and with a lot of attention to keep types well defined.19 20- **Highly generic** - no constraints on tokens, options (additional state data) and output types. Core module has not a single mention of strings as a part of normal flow. Some string-specific building blocks can be loaded from a separate module in case you need them.21 22- **Transparent**. Built on a very simple base idea - just a few type aliases. Whole parser state is accessible at any time.23 24- **Lightweight**. Zero dependencies. Just type aliases and functions.25 26- **Batteries included** - comes with a pretty big set of building blocks.27 28- **Easy to extend** - just follow the convention defined by type aliases when making your own building blocks. *(And maybe let me know what you think can be universally useful to be included in the package itself.)*29 30- **Easy to make configurable parsers**. Rather than dynamically composing parsers based on options or manually weaving options into a dynamic parser state, this package offers a standard way to treat options as a part of static data and access them at any moment for course correction.31 32- **Well tested** - comes with tests for everything including examples.33 34- **Practicality over "purity"**. To be understandable and self-consistent is more important than to follow an established encoding of abstract ideas. More on this below.35 36- **No streaming** - accepts a fixed array of tokens. It is simple, whole input can be accessed at any time if needed. More on this below.37 38- **Bring your own lexer/tokenizer** - if you need it. It doesn't matter how tokens are made - this package can consume anything you can type. I have a lexer as well, called [leac](https://github.com/mxxii/leac), and it is used in some examples, but there is nothing special about it to make it the best match (well, maybe the fact it is written in TypeScript, has equal level of maintenance and is made with arrays instead of iterators in mind as well).39 40 41## Changelog42 43Available here: [CHANGELOG.md](https://github.com/mxxii/peberminta/blob/main/CHANGELOG.md)44 45 46## Install47 48### Node49 50```shell51> npm i peberminta52```53 54```ts55import * as p from 'peberminta';56import * as pc from 'peberminta/char';57```58 59### Deno60 61```ts62import * as p from 'https://deno.land/x/peberminta@.../core.ts';63import * as pc from 'https://deno.land/x/peberminta@.../char.ts';64```65 66 67## Examples68 69- [JSON](https://github.com/mxxii/peberminta/blob/main/examples/json.ts);70- [CSV](https://github.com/mxxii/peberminta/blob/main/examples/csv.ts);71- [Hex Color](https://github.com/mxxii/peberminta/blob/main/examples/hexColor.ts);72- [Calc](https://github.com/mxxii/peberminta/blob/main/examples/calc.ts);73- [Brainfuck](https://github.com/mxxii/peberminta/blob/main/examples/bf1.ts) (and [another implementation](https://github.com/mxxii/peberminta/blob/main/examples/bf2.ts));74- [Non-decreasing sequences](https://github.com/mxxii/peberminta/blob/main/examples/nonDec.ts);75- *feel free to PR or request interesting compact grammar examples.*76 77### Published packages using `peberminta`78 79- [aspargvs](https://github.com/mxxii/aspargvs) - arg parser, CLI wrapper80- [parseley](https://github.com/mxxii/parseley) - CSS selectors parser81 82 83## API84 85Detailed API documentation with navigation and search:86 87- [core module](https://mxxii.github.io/peberminta/modules/core.html);88- [char module](https://mxxii.github.io/peberminta/modules/char.html).89 90### Convention91 92Whole package is built around these type aliases:93 94```typescript95export type Data<TToken,TOptions> = {96 tokens: TToken[],97 options: TOptions98};99 100export type Parser<TToken,TOptions,TValue> =101 (data: Data<TToken,TOptions>, i: number) => Result<TValue>;102 103export type Matcher<TToken,TOptions,TValue> =104 (data: Data<TToken,TOptions>, i: number) => Match<TValue>;105 106export type Result<TValue> = Match<TValue> | NonMatch;107 108export type Match<TValue> = {109 matched: true,110 position: number,111 value: TValue112};113 114export type NonMatch = {115 matched: false116};117```118 119- **Data** object holds tokens array and possibly an options object - it's just a container for all static data used by a parser. Parser position, on the other hand, has it's own life cycle and passed around separately.120 121- A **Parser** is a function that accepts Data object and a parser position, looks into the tokens array at the given position and returns either a Match with a parsed value (use `null` if there is no value) and a new position or a NonMatch.122 123- A **Matcher** is a special case of Parser that never fails and always returns a Match.124 125- **Result** object from a Parser can be either a Match or a NonMatch.126 127- **Match** is a result of successful parsing - it contains a parsed value and a new parser position.128 129- **NonMatch** is a result of unsuccessful parsing. It doesn't have any data attached to it.130 131- **TToken** can be any type.132 133- **TOptions** can be any type. Use it to make your parser customizable. Or set it as `undefined` and type as `unknown` if not needed.134 135### Building blocks136 137#### Core blocks138 139<div class="headlessTable">140 141| <!-- --> | <!-- --> | <!-- --> | <!-- -->142| -------- | -------- | -------- | --------143| [ab](https://mxxii.github.io/peberminta/modules/core.html#ab) | [abc](https://mxxii.github.io/peberminta/modules/core.html#abc) | [action](https://mxxii.github.io/peberminta/modules/core.html#action) | [ahead](https://mxxii.github.io/peberminta/modules/core.html#ahead)144| [all](https://mxxii.github.io/peberminta/modules/core.html#all) | _[and](https://mxxii.github.io/peberminta/modules/core.html#and)_ | [any](https://mxxii.github.io/peberminta/modules/core.html#any) | [chain](https://mxxii.github.io/peberminta/modules/core.html#chain)145| [chainReduce](https://mxxii.github.io/peberminta/modules/core.html#chainReduce) | [choice](https://mxxii.github.io/peberminta/modules/core.html#choice) | [condition](https://mxxii.github.io/peberminta/modules/core.html#condition) | [decide](https://mxxii.github.io/peberminta/modules/core.html#decide)146| _[discard](https://mxxii.github.io/peberminta/modules/core.html#discard)_ | _[eitherOr](https://mxxii.github.io/peberminta/modules/core.html#eitherOr)_ | [emit](https://mxxii.github.io/peberminta/modules/core.html#emit) | [end](https://mxxii.github.io/peberminta/modules/core.html#end)147| _[eof](https://mxxii.github.io/peberminta/modules/core.html#eof)_ | [error](https://mxxii.github.io/peberminta/modules/core.html#error) | [fail](https://mxxii.github.io/peberminta/modules/core.html#fail) | [flatten](https://mxxii.github.io/peberminta/modules/core.html#flatten)148| [flatten1](https://mxxii.github.io/peberminta/modules/core.html#flatten1) | [left](https://mxxii.github.io/peberminta/modules/core.html#left) | [leftAssoc1](https://mxxii.github.io/peberminta/modules/core.html#leftAssoc1) | [leftAssoc2](https://mxxii.github.io/peberminta/modules/core.html#leftAssoc2)149| [longest](https://mxxii.github.io/peberminta/modules/core.html#longest) | _[lookAhead](https://mxxii.github.io/peberminta/modules/core.html#lookAhead)_ | [make](https://mxxii.github.io/peberminta/modules/core.html#make) | [many](https://mxxii.github.io/peberminta/modules/core.html#many)150| [many1](https://mxxii.github.io/peberminta/modules/core.html#many1) | [map](https://mxxii.github.io/peberminta/modules/core.html#map) | [map1](https://mxxii.github.io/peberminta/modules/core.html#map1) | [middle](https://mxxii.github.io/peberminta/modules/core.html#middle)151| [not](https://mxxii.github.io/peberminta/modules/core.html#not) | _[of](https://mxxii.github.io/peberminta/modules/core.html#of)_ | [option](https://mxxii.github.io/peberminta/modules/core.html#option) | _[or](https://mxxii.github.io/peberminta/modules/core.html#or)_152| [otherwise](https://mxxii.github.io/peberminta/modules/core.html#otherwise) | [peek](https://mxxii.github.io/peberminta/modules/core.html#peek) | [recursive](https://mxxii.github.io/peberminta/modules/core.html#recursive) | [reduceLeft](https://mxxii.github.io/peberminta/modules/core.html#reduceLeft)153| [reduceRight](https://mxxii.github.io/peberminta/modules/core.html#reduceRight) | [right](https://mxxii.github.io/peberminta/modules/core.html#right) | [rightAssoc1](https://mxxii.github.io/peberminta/modules/core.html#rightAssoc1) | [rightAssoc2](https://mxxii.github.io/peberminta/modules/core.html#rightAssoc2)154| [satisfy](https://mxxii.github.io/peberminta/modules/core.html#satisfy) | [sepBy](https://mxxii.github.io/peberminta/modules/core.html#sepBy) | [sepBy1](https://mxxii.github.io/peberminta/modules/core.html#sepBy1) | [skip](https://mxxii.github.io/peberminta/modules/core.html#skip)155| _[some](https://mxxii.github.io/peberminta/modules/core.html#some)_ | [start](https://mxxii.github.io/peberminta/modules/core.html#start) | [takeUntil](https://mxxii.github.io/peberminta/modules/core.html#takeUntil) | [takeUntilP](https://mxxii.github.io/peberminta/modules/core.html#takeUntilP)156| [takeWhile](https://mxxii.github.io/peberminta/modules/core.html#takeWhile) | [takeWhileP](https://mxxii.github.io/peberminta/modules/core.html#takeWhileP) | [token](https://mxxii.github.io/peberminta/modules/core.html#token)157 158</div>159 160#### Core utilities161 162<div class="headlessTable">163 164| <!-- --> | <!-- --> | <!-- --> | <!-- -->165| -------- | -------- | -------- | --------166| [match](https://mxxii.github.io/peberminta/modules/core.html#match) | [parse](https://mxxii.github.io/peberminta/modules/core.html#parse) | [parserPosition](https://mxxii.github.io/peberminta/modules/core.html#parserPosition) | [remainingTokensNumber](https://mxxii.github.io/peberminta/modules/core.html#remainingTokensNumber)167| [tryParse](https://mxxii.github.io/peberminta/modules/char.html#tryParse)168 169</div>170 171#### Char blocks172 173<div class="headlessTable">174 175| <!-- --> | <!-- --> | <!-- --> | <!-- -->176| -------- | -------- | -------- | --------177| _[anyOf](https://mxxii.github.io/peberminta/modules/char.html#anyOf)_ | [char](https://mxxii.github.io/peberminta/modules/char.html#char) | [charTest](https://mxxii.github.io/peberminta/modules/char.html#charTest) | [concat](https://mxxii.github.io/peberminta/modules/char.html#concat)178| [noneOf](https://mxxii.github.io/peberminta/modules/char.html#noneOf) | [oneOf](https://mxxii.github.io/peberminta/modules/char.html#oneOf) | [str](https://mxxii.github.io/peberminta/modules/char.html#str)179 180</div>181 182#### Char utilities183 184<div class="headlessTable">185 186| <!-- --> | <!-- --> | <!-- --> | <!-- -->187| -------- | -------- | -------- | --------188| [match](https://mxxii.github.io/peberminta/modules/char.html#match) | [parse](https://mxxii.github.io/peberminta/modules/char.html#parse) | [parserPosition](https://mxxii.github.io/peberminta/modules/char.html#parserPosition) | [tryParse](https://mxxii.github.io/peberminta/modules/char.html#tryParse)189 190</div>191 192 193## What about ...?194 195- performance - The code is very simple but I won't put any unverified assumptions here. I'd be grateful to anyone who can set up a good benchmark project to compare different parser combinators.196 197- stable release - Current release is well thought out and tested. I leave a chance that some supplied functions may need an incompatible change. Before version 1.0.0 this will be done without a deprecation cycle.198 199- streams/iterators - Maybe some day, if the need to parse a stream of non-string data arise. For now I don't have a task that would force me to think well on how to design it. It would require a significant trade off and may end up being a separate module (like `char`) at best or even a separate package.200 201- Fantasy Land - You can find some familiar ideas here, especially when compared to Static Land. But I'm not concerned about compatibility with that spec - see "Practicality over "purity"" entry above. What I think might make sense is to add separate tests for laws applicable in context of this package. Low priority though.202 203 204## Some other parser combinator packages205 206- [arcsecond](https://github.com/francisrstokes/arcsecond);207- [parsimmon](https://github.com/jneen/parsimmon);208- [chevrotain](https://github.com/Chevrotain/chevrotain);209- [prsc.js](https://github.com/bwrrp/prsc.js);210- [lop](https://github.com/mwilliamson/lop);211- [parser-lang](https://github.com/disnet/parser-lang);212- *and more, with varied level of maintenance.*213