JavaScript Iterators and AsyncIterators for all!

iterall provides a few crucial utilities for implementing and working with
Iterables, Async Iterables and
Array-likes in all JavaScript environments, even old versions of
Internet Explorer, in a tiny library weighing well under 1KB when minified
and gzipped.

This is a library for libraries. If your library takes Arrays as input, accept
Iterables instead. If your library implements a new data-structure, make
it Iterable.

When installed via npm, iterall comes complete with Flow and
TypeScript definition files. Don't want to take the dependency? Feel free to
copy code directly from this repository.

// Limited to only Arrays ?
if (Array.isArray(thing)) {
  thing.forEach(function (item, i) {
    console.log('Index: ' + i, item)
  })
}

// Accepts all Iterables and Array-likes, in any JavaScript environment! ?
var isCollection = require('iterall').isCollection
var forEach = require('iterall').forEach

if (isCollection(thing)) {
  forEach(thing, function (item, i) {
    console.log('Index: ' + i, item)
  })
}

// Accepts all AsyncIterators, in any JavaScript environment! ⏳
var forAwaitEach = require('iterall').forAwaitEach

forAwaitEach(thing, function (item, i) {
  console.log('Index: ' + i, item)
}).then(function () {
  console.log('Done')
})

Why use Iterators?

For most of JavaScript's history it has provided two collection data-structures:
the Object and the Array. These collections can conceptually describe nearly
all data and so it's no suprise that libraries expecting lists of
things standardized on expecting and checking for an Array. This pattern even
resulted in the addition of a new method in ES5: Array.isArray().

As JavaScript applications grew in complexity, moved to the server
where CPU is a constrained resource, faced new problems and implemented new
algorithms, new data-structures are often required. With options from
linked lists to HAMTs developers can use what is most
efficient and provides the right properties for their program.

However none of these new data-structures can be used in libraries where an
Array is expected, which means developers are often stuck between abandoning
their favorite libraries or limiting their data-structure choices at the cost of
efficiency or usefulness.

To enable many related data-structures to be used interchangably we need a
protocol, and luckily for us ES2015 introduced the
Iteration Protocols to describe all list-like data-structures which
can be iterated. That includes not just the new-to-ES2015 Map and Set
collections but also existing ones like arguments, NodeList and the
various TypedArray, all of which return false for Array.isArray()
and in ES2015 implement the Iterator protocol.

While Iterators are defined in ES2015, they do not require ES2015 to work
correctly. In fact, Iterators were first introduced in 2012 in Firefox v17. Rather than using Symbol.iterator, they used the property name "@@iterator" (in fact, the ECMAScript
spec still refers to well-known Symbols using this @@ shorthand). By falling
back to use "@@iterator" when Symbol.iterator is not defined, Iterators can
be both safely defined and used by any version of JavaScript.

Not only were Iterables defined in ES2015, they were also implemented by the
built-in data-structures including Array. Older JavaScript
environments do not implement Array.prototype[@@iterator](), however this is
only a minor problem. JavaScript has another related and much older protocol:
Array-like. A value is "Array-like" if it has a numeric length property and
indexed access, but does not necessarily have methods like .push() or .forEach().
Much like Array.from, iterall's forEach() and
createIterator() methods also accept collections which are not Iterable but
are Array-like. This means that iterall can be used with Array,
arguments, NodeList, TypedArray and other Array-like collections
regardless of the JavaScript environment.

When libraries only accept Arrays as input, they stick developers with a tough
choice: limit which data-structures can be used or limit the ability to use that
library. Accepting Iterables removes this false dichotomy, and allows libraries
to be more generally useful. There's no need to limit to ES2015 environments and
bleeding-edge browsers to accept Iterable.

Only using Arrays can limit the efficiency and usefulness of your application
code, but custom data-structures can often feel like a fish out of water in
JavaScript programs, only working with code written specifically for it.
Protocols like Iterable helps these new data-structures work with more
libraries and built-in JavaScript behavior. There's no need to limit to ES2015
environments and bleeding-edge browsers to implement Iterable.

Why use AsyncIterators?

In the same way that Iterator provides a common interface for accessing many
different kinds of data-structures, AsyncIterator provides a common interface
over an asynchronous sequence of values (similar to Stream or Observable).

Async Iterators are not yet an official part of JavaScript, however they're
a "Stage 3" proposal to be added, and browser vendors are
working on adding support.
However, Async Iterators can be both safely defined and used today by
any version of JavaScript, by using the utilities in iterall.

FAQ

Aren't Iterables slower than Arrays? I want the highest performance possible.

Arrays are Iterables. Iterable is a protocol that Arrays adhere to in ES2015.
It's true that creating an Iterator and stepping through it can present some
overhead compared to a simple for-loop or array.forEach. However iterall's
forEach will delegate directly to array.forEach and will use a for-loop for
Array-like objects, ensuring the best performance for Arrays while still
maintaining support for all Iterables.

Should my library functions also return Iterables instead of Arrays? Won't
that be limiting?

That could definitely be limiting if you return some generic Iterable where you
could have returned an Array, and (depending on context) I wouldn't recommend
you stop returning Arrays from functions if that's what you're doing today.
However if your functions are returning some collection data-structure that is
not an Array, you should certainly consider having them implement the
Iterable protocol so they can be more widely useful.

Here are a few examples:

In React, render functions are expected to return view trees, where any
node (e.g. a <ul>) can have many children (e.g. many <li>). While it could
expect those children to always be represented as an Array, that would limit
React's usefulness - other data-structures couldn't be used. Instead, React
expects those children to be represented as an Iterable. That allows it to
continue to accept Arrays, but also accept many other data-structures.

Immutable.js implements many new kinds of data-structures (including HAMT)
all of which implement Iterable, which allows them to be used in many of
JavaScript's built-in functions, but also allows them to be used by many
libraries which accept Iterables, including React. Also, similar to
Array.from, Immutable.js's constructors accept not only Arrays,
but any Iterable, allowing you to build any of these new data-structures from
any other data-structure.

Where are all the other functions like map, filter, and reduce?

Those "higher order" collection functions are awesome, but they don't belong in
this library. Instead this library should be used as a basis for building such
a library (as it should be used for many other libraries). The forEach
function provided by iterall can be used as the underpinning for these.

As an example:

function reduce (collection, reducer, initial) {
  var reduced = initial
  forEach(collection, function (item) {
    reduced = reducer(reduced, item)
  })
  return reduced
}

How do I break out of a forEach or forAwaitEach loop early?

While for of and for await of loops allow breaking out of a loop early with
a break statement, the forEach() and forAwaitEach() functions (much like
Array's forEach) do not support early breaking.

Similar to the "higher order" functions described above, this library can be the
basis for this extended behavior. To support early break outs, you can use a
wrapping function supporting early breaking by throwing a BREAK sentinel value
from the callback and using a try/catch block to early break:

const BREAK = {}

function forEachBreakable (collection, callback) {
  try {
    forEach(collection, callback)
  } catch (error) {
    if (error !== BREAK) {
      throw error
    }
  }
}

async function forAwaitEachBreakable (collection, callback) {
  try {
    await forAwaitEach(collection, callback)
  } catch (error) {
    if (error !== BREAK) {
      throw error
    }
  }
}

// Example usages:
forEachBreakable(obj, function (value) {
  if (shouldBreakOn(value)) {
    throw BREAK
  }
  console.log(value)
})

forAwaitEachBreakable(obj, async function (value) {
  if (await shouldBreakOn(value)) {
    throw BREAK
  }
  console.log(value)
})

Note: This technique also works with the native Array forEach method!

API

Table of Contents

• Iterable
• Iterator
• $$iterator
  • Examples
• isIterable
  • Parameters
  • Examples
• isArrayLike
  • Parameters
  • Examples
• isCollection
  • Parameters
  • Examples
• getIterator
  • Parameters
  • Examples
• getIteratorMethod
  • Parameters
  • Examples
• createIterator
  • Parameters
  • Examples
• forEach
  • Parameters
  • Examples
• AsyncIterable
• AsyncIterator
• $$asyncIterator
  • Examples
• isAsyncIterable
  • Parameters
  • Examples
• getAsyncIterator
  • Parameters
  • Examples
• getAsyncIteratorMethod
  • Parameters
  • Examples
• createAsyncIterator
  • Parameters
  • Examples
• forAwaitEach
  • Parameters
  • Examples

Iterable

• See: MDN Iteration protocols

Iterable
is a protocol which when implemented allows a JavaScript object to define
their iteration behavior, such as what values are looped over in a
for...of
loop or iterall's forEach function. Many built-in types
implement the Iterable protocol, including Array and Map.

While described by the ES2015 version of JavaScript
it can be utilized by any version of JavaScript.

Iterator

• See: MDN Iteration protocols

Iterator
is a protocol which describes a standard way to produce a sequence of
values, typically the values of the Iterable represented by this Iterator.

While described by the ES2015 version of JavaScript
it can be utilized by any version of JavaScript.

$$iterator

A property name to be used as the name of an Iterable's method responsible
for producing an Iterator, referred to as @@iterator. Typically represents
the value Symbol.iterator but falls back to the string "@@iterator" when
Symbol.iterator is not defined.

Use $$iterator for defining new Iterables instead of Symbol.iterator,
but do not use it for accessing existing Iterables, instead use
getIterator or isIterable.

Type: (Symbol , string)

Examples

var $$iterator = require('iterall').$$iterator

function Counter (to) {
  this.to = to
}

Counter.prototype[$$iterator] = function () {
  return {
    to: this.to,
    num: 0,
    next () {
      if (this.num >= this.to) {
        return { value: undefined, done: true }
      }
      return { value: this.num++, done: false }
    }
  }
}

var counter = new Counter(3)
for (var number of counter) {
  console.log(number) // 0 ... 1 ... 2
}

isIterable

Returns true if the provided object implements the Iterator protocol via
either implementing a Symbol.iterator or "@@iterator" method.

Parameters

• obj A value which might implement the Iterable protocol.

Examples

var isIterable = require('iterall').isIterable
isIterable([ 1, 2, 3 ]) // true
isIterable('ABC') // true
isIterable({ length: 1, 0: 'Alpha' }) // false
isIterable({ key: 'value' }) // false
isIterable(new Map()) // true

Returns boolean true if Iterable.

isArrayLike

Returns true if the provided object implements the Array-like protocol via
defining a positive-integer length property.

Parameters

• obj A value which might implement the Array-like protocol.

Examples

var isArrayLike = require('iterall').isArrayLike
isArrayLike([ 1, 2, 3 ]) // true
isArrayLike('ABC') // true
isArrayLike({ length: 1, 0: 'Alpha' }) // true
isArrayLike({ key: 'value' }) // false
isArrayLike(new Map()) // false

Returns boolean true if Array-like.

isCollection

Returns true if the provided object is an Object (i.e. not a string literal)
and is either Iterable or Array-like.

This may be used in place of Array.isArray() to determine if an
object should be iterated-over. It always excludes string literals and
includes Arrays (regardless of if it is Iterable). It also includes other
Array-like objects such as NodeList, TypedArray, and Buffer.

Parameters

• obj An Object value which might implement the Iterable or Array-like protocols.

Examples

var isCollection = require('iterall').isCollection
isCollection([ 1, 2, 3 ]) // true
isCollection('ABC') // false
isCollection({ length: 1, 0: 'Alpha' }) // true
isCollection({ key: 'value' }) // false
isCollection(new Map()) // true

var forEach = require('iterall').forEach
if (isCollection(obj)) {
  forEach(obj, function (value) {
    console.log(value)
  })
}

Returns boolean true if Iterable or Array-like Object.

getIterator

If the provided object implements the Iterator protocol, its Iterator object
is returned. Otherwise returns undefined.

Parameters

• iterable Iterable<T> An Iterable object which is the source of an Iterator.

Examples

var getIterator = require('iterall').getIterator
var iterator = getIterator([ 1, 2, 3 ])
iterator.next() // { value: 1, done: false }
iterator.next() // { value: 2, done: false }
iterator.next() // { value: 3, done: false }
iterator.next() // { value: undefined, done: true }

Returns Iterator<T> new Iterator instance.

getIteratorMethod

If the provided object implements the Iterator protocol, the method
responsible for producing its Iterator object is returned.

This is used in rare cases for performance tuning. This method must be called
with obj as the contextual this-argument.

Parameters

• iterable Iterable<T> An Iterable object which defines an @@iterator method.

Examples

var getIteratorMethod = require('iterall').getIteratorMethod
var myArray = [ 1, 2, 3 ]
var method = getIteratorMethod(myArray)
if (method) {
  var iterator = method.call(myArray)
}

Returns function (): Iterator<T> @@iterator method.

createIterator

Similar to getIterator, this method returns a new Iterator given an
Iterable. However it will also create an Iterator for a non-Iterable
Array-like collection, such as Array in a non-ES2015 environment.

createIterator is complimentary to forEach, but allows a "pull"-based
iteration as opposed to forEach's "push"-based iteration.

createIterator produces an Iterator for Array-likes with the same behavior
as ArrayIteratorPrototype described in the ECMAScript specification, and
does not skip over "holes".

Parameters

• collection (Iterable<T>, {length: number}) An Iterable or Array-like object to produce an Iterator.

Examples

var createIterator = require('iterall').createIterator

var myArraylike = { length: 3, 0: 'Alpha', 1: 'Bravo', 2: 'Charlie' }
var iterator = createIterator(myArraylike)
iterator.next() // { value: 'Alpha', done: false }
iterator.next() // { value: 'Bravo', done: false }
iterator.next() // { value: 'Charlie', done: false }
iterator.next() // { value: undefined, done: true }

Returns Iterator<T> new Iterator instance.

forEach

Given an object which either implements the Iterable protocol or is
Array-like, iterate over it, calling the callback at each iteration.

Use forEach where you would expect to use a for ... of loop in ES6.
However forEach adheres to the behavior of Array#forEach described in
the ECMAScript specification, skipping over "holes" in Array-likes. It will
also delegate to a forEach method on collection if one is defined,
ensuring native performance for Arrays.

Similar to Array#forEach, the callback function accepts three
arguments, and is provided with thisArg as the calling context.

Note: providing an infinite Iterator to forEach will produce an error.

Parameters

• collection (Iterable<T>, {length: number}) The Iterable or array to iterate over.
• callback function (T, number, object) Function to execute for each iteration, taking up to three arguments
• thisArg Optional. Value to use as this when executing callback.

Examples

var forEach = require('iterall').forEach

forEach(myIterable, function (value, index, iterable) {
  console.log(value, index, iterable === myIterable)
})

// ES6:
for (let value of myIterable) {
  console.log(value)
}

// Any JavaScript environment:
forEach(myIterable, function (value) {
  console.log(value)
})

AsyncIterable

• See: Async Iteration Proposal

AsyncIterable
is a protocol which when implemented allows a JavaScript object to define
an asynchronous iteration behavior, such as what values are looped over in
a for-await-of
loop or iterall's forAwaitEach function.

While described as a proposed addition to the ES2017 version of JavaScript
it can be utilized by any version of JavaScript.

AsyncIterator

• See: Async Iteration Proposal

AsyncIterator
is a protocol which describes a standard way to produce and consume an
asynchronous sequence of values, typically the values of the
AsyncIterable represented by this AsyncIterator.

AsyncIterator is similar to Observable or Stream. Like an Iterator it
also as a next() method, however instead of an IteratorResult,
calling this method returns a Promise for a IteratorResult.

While described as a proposed addition to the ES2017 version of JavaScript
it can be utilized by any version of JavaScript.

$$asyncIterator

A property name to be used as the name of an AsyncIterable's method
responsible for producing an Iterator, referred to as @@asyncIterator.
Typically represents the value Symbol.asyncIterator but falls back to the
string "@@asyncIterator" when Symbol.asyncIterator is not defined.

Use $$asyncIterator for defining new AsyncIterables instead of
Symbol.asyncIterator, but do not use it for accessing existing Iterables,
instead use getAsyncIterator or isAsyncIterable.

Type: (Symbol , string)

Examples

var $$asyncIterator = require('iterall').$$asyncIterator

function Chirper (to) {
  this.to = to
}

Chirper.prototype[$$asyncIterator] = function () {
  return {
    to: this.to,
    num: 0,
    next () {
      return new Promise(resolve => {
        if (this.num >= this.to) {
          resolve({ value: undefined, done: true })
        } else {
          setTimeout(() => {
            resolve({ value: this.num++, done: false })
          }, 1000)
        }
      })
    }
  }
}

var chirper = new Chirper(3)
for await (var number of chirper) {
  console.log(number) // 0 ...wait... 1 ...wait... 2
}

isAsyncIterable

Returns true if the provided object implements the AsyncIterator protocol via
either implementing a Symbol.asyncIterator or "@@asyncIterator" method.

Parameters

• obj A value which might implement the AsyncIterable protocol.

Examples

var isAsyncIterable = require('iterall').isAsyncIterable
isAsyncIterable(myStream) // true
isAsyncIterable('ABC') // false

Returns boolean true if AsyncIterable.

getAsyncIterator

If the provided object implements the AsyncIterator protocol, its
AsyncIterator object is returned. Otherwise returns undefined.

Parameters

• asyncIterable AsyncIterable<T> An AsyncIterable object which is the source of an AsyncIterator.

Examples

var getAsyncIterator = require('iterall').getAsyncIterator
var asyncIterator = getAsyncIterator(myStream)
asyncIterator.next().then(console.log) // { value: 1, done: false }
asyncIterator.next().then(console.log) // { value: 2, done: false }
asyncIterator.next().then(console.log) // { value: 3, done: false }
asyncIterator.next().then(console.log) // { value: undefined, done: true }

Returns AsyncIterator<T> new AsyncIterator instance.

getAsyncIteratorMethod

If the provided object implements the AsyncIterator protocol, the method
responsible for producing its AsyncIterator object is returned.

This is used in rare cases for performance tuning. This method must be called
with obj as the contextual this-argument.

Parameters

• asyncIterable AsyncIterable<T> An AsyncIterable object which defines an @@asyncIterator method.

Examples

var getAsyncIteratorMethod = require('iterall').getAsyncIteratorMethod
var method = getAsyncIteratorMethod(myStream)
if (method) {
  var asyncIterator = method.call(myStream)
}

Returns function (): AsyncIterator<T> @@asyncIterator method.

createAsyncIterator

Similar to getAsyncIterator, this method returns a new AsyncIterator
given an AsyncIterable. However it will also create an AsyncIterator for a
non-async Iterable as well as non-Iterable Array-like collection, such as
Array in a pre-ES2015 environment.

createAsyncIterator is complimentary to forAwaitEach, but allows a
buffering "pull"-based iteration as opposed to forAwaitEach's
"push"-based iteration.

createAsyncIterator produces an AsyncIterator for non-async Iterables as
described in the ECMAScript proposal Async-from-Sync Iterator Objects.

Note: Creating AsyncIterators requires the existence of Promise.
While Promise has been available in modern browsers for a number of
years, legacy browsers (like IE 11) may require a polyfill.

Parameters

• source (AsyncIterable<T>, Iterable<T>, {length: number}) An AsyncIterable, Iterable, or Array-like object to produce an Iterator.

Examples

var createAsyncIterator = require('iterall').createAsyncIterator

var myArraylike = { length: 3, 0: 'Alpha', 1: 'Bravo', 2: 'Charlie' }
var iterator = createAsyncIterator(myArraylike)
iterator.next().then(console.log) // { value: 'Alpha', done: false }
iterator.next().then(console.log) // { value: 'Bravo', done: false }
iterator.next().then(console.log) // { value: 'Charlie', done: false }
iterator.next().then(console.log) // { value: undefined, done: true }

Returns AsyncIterator<T> new AsyncIterator instance.

forAwaitEach

Given an object which either implements the AsyncIterable protocol or is
Array-like, iterate over it, calling the callback at each iteration.

Use forAwaitEach where you would expect to use a for-await-of loop.

Similar to Array#forEach, the callback function accepts three
arguments, and is provided with thisArg as the calling context.

Note: Using forAwaitEach requires the existence of Promise.
While Promise has been available in modern browsers for a number of
years, legacy browsers (like IE 11) may require a polyfill.

Parameters

• source (AsyncIterable<T>, Iterable<(Promise<T>, T)>, {length: number}) The AsyncIterable or array to iterate over.
• callback function (T, number, object) Function to execute for each iteration, taking up to three arguments
• thisArg Optional. Value to use as this when executing callback.

Examples

var forAwaitEach = require('iterall').forAwaitEach

forAwaitEach(myIterable, function (value, index, iterable) {
  console.log(value, index, iterable === myIterable)
})

// ES2017:
for await (let value of myAsyncIterable) {
  console.log(await doSomethingAsync(value))
}
console.log('done')

// Any JavaScript environment:
forAwaitEach(myAsyncIterable, function (value) {
  return doSomethingAsync(value).then(console.log)
}).then(function () {
  console.log('done')
})

Contributing

Contributions are welcome and encouraged!

Remember that this library is designed to be small, straight-forward, and
well-tested. The value of new additional features will be weighed against their
size. This library also seeks to leverage and mirror the
ECMAScript specification in its behavior as much as possible and reasonable.

This repository has far more documentation and explanation than code, and it is
expected that the majority of contributions will come in the form of improving
these.