AK-21/Graphite-Industrial-Intelligence
0
1# bl *(BufferList)*2 3[](https://travis-ci.com/rvagg/bl/)4 5**A Node.js Buffer list collector, reader and streamer thingy.**6 7[](https://nodei.co/npm/bl/)8 9**bl** is a storage object for collections of Node Buffers, exposing them with the main Buffer readable API. Also works as a duplex stream so you can collect buffers from a stream that emits them and emit buffers to a stream that consumes them!10 11The original buffers are kept intact and copies are only done as necessary. Any reads that require the use of a single original buffer will return a slice of that buffer only (which references the same memory as the original buffer). Reads that span buffers perform concatenation as required and return the results transparently.12 13```js14const { BufferList } = require('bl')15 16const bl = new BufferList()17bl.append(Buffer.from('abcd'))18bl.append(Buffer.from('efg'))19bl.append('hi') // bl will also accept & convert Strings20bl.append(Buffer.from('j'))21bl.append(Buffer.from([ 0x3, 0x4 ]))22 23console.log(bl.length) // 1224 25console.log(bl.slice(0, 10).toString('ascii')) // 'abcdefghij'26console.log(bl.slice(3, 10).toString('ascii')) // 'defghij'27console.log(bl.slice(3, 6).toString('ascii')) // 'def'28console.log(bl.slice(3, 8).toString('ascii')) // 'defgh'29console.log(bl.slice(5, 10).toString('ascii')) // 'fghij'30 31console.log(bl.indexOf('def')) // 332console.log(bl.indexOf('asdf')) // -133 34// or just use toString!35console.log(bl.toString()) // 'abcdefghij\u0003\u0004'36console.log(bl.toString('ascii', 3, 8)) // 'defgh'37console.log(bl.toString('ascii', 5, 10)) // 'fghij'38 39// other standard Buffer readables40console.log(bl.readUInt16BE(10)) // 0x030441console.log(bl.readUInt16LE(10)) // 0x040342```43 44Give it a callback in the constructor and use it just like **[concat-stream](https://github.com/maxogden/node-concat-stream)**:45 46```js47const { BufferListStream } = require('bl')48const fs = require('fs')49 50fs.createReadStream('README.md')51 .pipe(BufferListStream((err, data) => { // note 'new' isn't strictly required52 // `data` is a complete Buffer object containing the full data53 console.log(data.toString())54 }))55```56 57Note that when you use the *callback* method like this, the resulting `data` parameter is a concatenation of all `Buffer` objects in the list. If you want to avoid the overhead of this concatenation (in cases of extreme performance consciousness), then avoid the *callback* method and just listen to `'end'` instead, like a standard Stream.58 59Or to fetch a URL using [hyperquest](https://github.com/substack/hyperquest) (should work with [request](http://github.com/mikeal/request) and even plain Node http too!):60 61```js62const hyperquest = require('hyperquest')63const { BufferListStream } = require('bl')64 65const url = 'https://raw.github.com/rvagg/bl/master/README.md'66 67hyperquest(url).pipe(BufferListStream((err, data) => {68 console.log(data.toString())69}))70```71 72Or, use it as a readable stream to recompose a list of Buffers to an output source:73 74```js75const { BufferListStream } = require('bl')76const fs = require('fs')77 78var bl = new BufferListStream()79bl.append(Buffer.from('abcd'))80bl.append(Buffer.from('efg'))81bl.append(Buffer.from('hi'))82bl.append(Buffer.from('j'))83 84bl.pipe(fs.createWriteStream('gibberish.txt'))85```86 87## API88 89 * <a href="#ctor"><code><b>new BufferList([ buf ])</b></code></a>90 * <a href="#isBufferList"><code><b>BufferList.isBufferList(obj)</b></code></a>91 * <a href="#length"><code>bl.<b>length</b></code></a>92 * <a href="#append"><code>bl.<b>append(buffer)</b></code></a>93 * <a href="#get"><code>bl.<b>get(index)</b></code></a>94 * <a href="#indexOf"><code>bl.<b>indexOf(value[, byteOffset][, encoding])</b></code></a>95 * <a href="#slice"><code>bl.<b>slice([ start[, end ] ])</b></code></a>96 * <a href="#shallowSlice"><code>bl.<b>shallowSlice([ start[, end ] ])</b></code></a>97 * <a href="#copy"><code>bl.<b>copy(dest, [ destStart, [ srcStart [, srcEnd ] ] ])</b></code></a>98 * <a href="#duplicate"><code>bl.<b>duplicate()</b></code></a>99 * <a href="#consume"><code>bl.<b>consume(bytes)</b></code></a>100 * <a href="#toString"><code>bl.<b>toString([encoding, [ start, [ end ]]])</b></code></a>101 * <a href="#readXX"><code>bl.<b>readDoubleBE()</b></code>, <code>bl.<b>readDoubleLE()</b></code>, <code>bl.<b>readFloatBE()</b></code>, <code>bl.<b>readFloatLE()</b></code>, <code>bl.<b>readInt32BE()</b></code>, <code>bl.<b>readInt32LE()</b></code>, <code>bl.<b>readUInt32BE()</b></code>, <code>bl.<b>readUInt32LE()</b></code>, <code>bl.<b>readInt16BE()</b></code>, <code>bl.<b>readInt16LE()</b></code>, <code>bl.<b>readUInt16BE()</b></code>, <code>bl.<b>readUInt16LE()</b></code>, <code>bl.<b>readInt8()</b></code>, <code>bl.<b>readUInt8()</b></code></a>102 * <a href="#ctorStream"><code><b>new BufferListStream([ callback ])</b></code></a>103 104--------------------------------------------------------105<a name="ctor"></a>106### new BufferList([ Buffer | Buffer array | BufferList | BufferList array | String ])107No arguments are _required_ for the constructor, but you can initialise the list by passing in a single `Buffer` object or an array of `Buffer` objects.108 109`new` is not strictly required, if you don't instantiate a new object, it will be done automatically for you so you can create a new instance simply with:110 111```js112const { BufferList } = require('bl')113const bl = BufferList()114 115// equivalent to:116 117const { BufferList } = require('bl')118const bl = new BufferList()119```120 121--------------------------------------------------------122<a name="isBufferList"></a>123### BufferList.isBufferList(obj)124Determines if the passed object is a `BufferList`. It will return `true` if the passed object is an instance of `BufferList` **or** `BufferListStream` and `false` otherwise.125 126N.B. this won't return `true` for `BufferList` or `BufferListStream` instances created by versions of this library before this static method was added.127 128--------------------------------------------------------129<a name="length"></a>130### bl.length131Get the length of the list in bytes. This is the sum of the lengths of all of the buffers contained in the list, minus any initial offset for a semi-consumed buffer at the beginning. Should accurately represent the total number of bytes that can be read from the list.132 133--------------------------------------------------------134<a name="append"></a>135### bl.append(Buffer | Buffer array | BufferList | BufferList array | String)136`append(buffer)` adds an additional buffer or BufferList to the internal list. `this` is returned so it can be chained.137 138--------------------------------------------------------139<a name="get"></a>140### bl.get(index)141`get()` will return the byte at the specified index.142 143--------------------------------------------------------144<a name="indexOf"></a>145### bl.indexOf(value[, byteOffset][, encoding])146`get()` will return the byte at the specified index.147`indexOf()` method returns the first index at which a given element can be found in the BufferList, or -1 if it is not present.148 149--------------------------------------------------------150<a name="slice"></a>151### bl.slice([ start, [ end ] ])152`slice()` returns a new `Buffer` object containing the bytes within the range specified. Both `start` and `end` are optional and will default to the beginning and end of the list respectively.153 154If the requested range spans a single internal buffer then a slice of that buffer will be returned which shares the original memory range of that Buffer. If the range spans multiple buffers then copy operations will likely occur to give you a uniform Buffer.155 156--------------------------------------------------------157<a name="shallowSlice"></a>158### bl.shallowSlice([ start, [ end ] ])159`shallowSlice()` returns a new `BufferList` object containing the bytes within the range specified. Both `start` and `end` are optional and will default to the beginning and end of the list respectively.160 161No copies will be performed. All buffers in the result share memory with the original list.162 163--------------------------------------------------------164<a name="copy"></a>165### bl.copy(dest, [ destStart, [ srcStart [, srcEnd ] ] ])166`copy()` copies the content of the list in the `dest` buffer, starting from `destStart` and containing the bytes within the range specified with `srcStart` to `srcEnd`. `destStart`, `start` and `end` are optional and will default to the beginning of the `dest` buffer, and the beginning and end of the list respectively.167 168--------------------------------------------------------169<a name="duplicate"></a>170### bl.duplicate()171`duplicate()` performs a **shallow-copy** of the list. The internal Buffers remains the same, so if you change the underlying Buffers, the change will be reflected in both the original and the duplicate. This method is needed if you want to call `consume()` or `pipe()` and still keep the original list.Example:172 173```js174var bl = new BufferListStream()175 176bl.append('hello')177bl.append(' world')178bl.append('\n')179 180bl.duplicate().pipe(process.stdout, { end: false })181 182console.log(bl.toString())183```184 185--------------------------------------------------------186<a name="consume"></a>187### bl.consume(bytes)188`consume()` will shift bytes *off the start of the list*. The number of bytes consumed don't need to line up with the sizes of the internal Buffers—initial offsets will be calculated accordingly in order to give you a consistent view of the data.189 190--------------------------------------------------------191<a name="toString"></a>192### bl.toString([encoding, [ start, [ end ]]])193`toString()` will return a string representation of the buffer. The optional `start` and `end` arguments are passed on to `slice()`, while the `encoding` is passed on to `toString()` of the resulting Buffer. See the [Buffer#toString()](http://nodejs.org/docs/latest/api/buffer.html#buffer_buf_tostring_encoding_start_end) documentation for more information.194 195--------------------------------------------------------196<a name="readXX"></a>197### bl.readDoubleBE(), bl.readDoubleLE(), bl.readFloatBE(), bl.readFloatLE(), bl.readInt32BE(), bl.readInt32LE(), bl.readUInt32BE(), bl.readUInt32LE(), bl.readInt16BE(), bl.readInt16LE(), bl.readUInt16BE(), bl.readUInt16LE(), bl.readInt8(), bl.readUInt8()198 199All of the standard byte-reading methods of the `Buffer` interface are implemented and will operate across internal Buffer boundaries transparently.200 201See the <b><code>[Buffer](http://nodejs.org/docs/latest/api/buffer.html)</code></b> documentation for how these work.202 203--------------------------------------------------------204<a name="ctorStream"></a>205### new BufferListStream([ callback | Buffer | Buffer array | BufferList | BufferList array | String ])206**BufferListStream** is a Node **[Duplex Stream](http://nodejs.org/docs/latest/api/stream.html#stream_class_stream_duplex)**, so it can be read from and written to like a standard Node stream. You can also `pipe()` to and from a **BufferListStream** instance.207 208The constructor takes an optional callback, if supplied, the callback will be called with an error argument followed by a reference to the **bl** instance, when `bl.end()` is called (i.e. from a piped stream). This is a convenient method of collecting the entire contents of a stream, particularly when the stream is *chunky*, such as a network stream.209 210Normally, no arguments are required for the constructor, but you can initialise the list by passing in a single `Buffer` object or an array of `Buffer` object.211 212`new` is not strictly required, if you don't instantiate a new object, it will be done automatically for you so you can create a new instance simply with:213 214```js215const { BufferListStream } = require('bl')216const bl = BufferListStream()217 218// equivalent to:219 220const { BufferListStream } = require('bl')221const bl = new BufferListStream()222```223 224N.B. For backwards compatibility reasons, `BufferListStream` is the **default** export when you `require('bl')`:225 226```js227const { BufferListStream } = require('bl')228// equivalent to:229const BufferListStream = require('bl')230```231 232--------------------------------------------------------233 234## Contributors235 236**bl** is brought to you by the following hackers:237 238 * [Rod Vagg](https://github.com/rvagg)239 * [Matteo Collina](https://github.com/mcollina)240 * [Jarett Cruger](https://github.com/jcrugzz)241 242<a name="license"></a>243## License & copyright244 245Copyright (c) 2013-2019 bl contributors (listed above).246 247bl is licensed under the MIT license. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE.md file for more details.248