Iterators, For Of Loop, Custom Iterators, Generators and a taste of Functional JavaScript in -ECMA2015 (ES6)

Custom Iterators are here! Go functional!

Posted by Anas R Firdousi on April 29th, 2016
15-22 mins read

Introduction

In our last post, we discussed first 6 things you can start with learning ES6. In this post, we will discuss a lot of exciting ES6 features and if you are into Functional JavaScript or would like to dive deep into it, you may love this post. Iterators are now a real thing, a standard, an actual JavaScript construct. Other languages have these features for a long time but they were missing in a powerful language like JavaScript. One more thing, before we jump onto iterators, there is a bonus starter kit in this post. The all new arrow syntax. We will discuss this new arrow syntax real quick and then move to iterators! If you think you can skip the arrow section, feel free to jump directly onto the Iterator Section.

Quick Intro to Arrow (Fat Arrow/Lambda Arrow > Whatever!)

ES6 uses the term Arrow functions so lets simply call these arrow functions. Quick comparative first look, functions in ES5 and functions in ES6:



  //ES5 Style  
  function sum(x,y){
    return x+y;
  }

  //ES6 Style
  let sum = (x,y) => x+y;


How cool is the ES6 style! Less ceremony, no return statement, no curly braces, no "function" keyword! Result of any "calculation statement" like x*y or x/y will just be returned so you don't have to specify a "return" there. Left hand side of => (x,y) are the function parameters and the right hand side of => is the function body, the code we want to execute i.e x+y in this case. Event less ceremony if your functions accepts only 1 parameter or no parameter. Check this out:



  //ES6 Style
  //Example#1

  let timesTwo = x => x*2;   // No need of () around x. 
  console.log(timesTwo(10)); //20

  //Example#2
  let getID = x => `ABC-${x}`;  // No need of () around x. 
  console.log(getID(10));    // ABC-10

  //Example#3
  let getVersion = () => "v0.0.1";
  console.log(getVersion()); //v0.0.1



What if our function has more than one statement? You need to use curly brackets and also a "return" statement in case your function returns something.



  //ES6 Style
  //Example#1

  let learnES6 = () => {
        
        console.log("Started learning...");
        console.log("Halfway...");
        console.log("Done...");

        return true;

  }
  console.log(learnES6());
    
  //Started learning...
  //Halfway...
  //Done...
  //true


Can Arrow functions help on methods where I have provide a function to run over each item in the list, for example the forEach method.


  
  var points = [10,20,30];

  //ES5 Style

  var result = 1;
  points.forEach(function(p){
    result = result * p
  });
  console.log(result); // 6000

  //ES6 Style

  let points = [10,20,30];
  let result = 1;
  points.forEach(p => result *= p ); // How expressive is this!!!
  console.log(result); // 6000


  //Feel free to replace forEach with other array methods like .map/.filter e.t.c for
  //other specific use cases.


One last thing on arrow which is super useful! In ES5, do you remember doing var self= this; or var that = this; or using bind() whenever we wanted to save a reference to this or use a different reference to "this", which changes in different scenarios. For example, in a setTimeInterval or within a forEach loop and several other places, the meaning of this changes which was a big pain.That used to happen because JavaScript used to form a different closure around your running code and changed context of "this". We had to maintain a reference to point to the actual "this". But story of dealing with "this" in JavaScript is now old! Congratulations, arrow functions remember your this and you don't have to do all the crazy stuff! Quick example?


  function getData(myDataService) {
        
        this.spec = "";

        myDataService
            .getData()
            .then(function(result) {
                this.spec = result;
            });
  }

The above code demonstrate the use of a async call using Promises. We know that "then()" is called asynchronously whenever getData() http call recieves the response. The problem is when we do

this.spec = result;
and you may get something like :


  Uncaught ReferenceError: spec is not defined

Why? because then() created a closure around and "spec" is not visible. Let's look at how we used to solve this in ES5 and how arrow function can help us here:


 //Solution in ES5
 function getData(myDataService) {
        var self = this;        
        this.spec = "";

        myDataService
            .getData()
            .then(function(result) {
                self.spec = result;
            });
  }

  //Solution in ES6 ( ECMA2015 ) 
 function getData(myDataService) {
        
        this.spec = "";
        myDataService
            .getData()
            .then((result) => {
                this.spec = result;  // No self= this! No that=this! Yayy!!!
            });
  }

Oh don't you love this! No unnecessary closures around async call backs & consequently we don't need to save our original "this" where async call was made.

This is extremely powerful in situations where unnecessary closures are created for example while using functions from utility libraries like 'lodash'.

The above intro, by no means is a complete intro to Arrow but just a quick recap before we jump to Iterators. We will do Arrows only in a separate post.

Iterables & Iterators

Iterator and Iterables are finally standardized in JavaScript. Here is the idea behind it:

  • Assume you have a collection of items, now a collection of items can be anything, an array of numbers, an array of objects, a set, a map and infact collections can also be object computed on the fly or even a custom data structure like a linked list, a tree or almost any other data structure. So a collection for us is a sequence of any kind of objects.
    We won't talk about "Set", "Map", "Weak Map" in JavaScript. Those are new to ES6 too but out of the scope of this post.
  • A collection can or can not be a iterable.
  • If a collection is a "Iterable" I can get a "Iterator" of the collection.
  • A "Iterator" is an object with a next() method and helps you walk the collection, one item at a time. This One Item At A Time is important here.
  • Every time you call the next() method of an Iterator, it gives you two things.
    • A value
    • A indicator/a flag indicating if the collection has more values or the one we just read was the last item in collection.
  • What's cool about Iterators? Iterator is a solid abstraction over the "type of collection" we are traversing, because, we do not have to worry about it. Have you thought about the "Iterator" pattern in software design pattern terms (not ES6 term) ? Iterator pattern provides you an abstraction for the traversal of different data structures without exposing its underlying representation. If you are excited about Iterator Pattern in terms of design patterns, we discussed that in detail in one of our posts on Observer and Iterator Pattern. Click here to read that article
  • All what we need, is to call Iterators's next() method to get the next item of the collection. The collection be an array, or map or even the DOM tree. Who cares!
  • We get a "Iterator" out of a "Iterable" by calling values() function on the iterable.
Picture worth a thousand word!
Please do not take any inspiration on how variables and functions are named in the post. Use proper naming conventions in your real world apps.

Comparing Iterator with others

Iterators are an abstraction and abstraction do take away some features that you might expect in an underlying type. For example, an Array in JavaScript has length property but Iterators do not. So I can never know how many items an Iterator might return until I traverse values one by one and get to the done:true flag indicating an end to my collection. Now at the same time when an abstraction take away some features, we get some flexibility which we might not have in an underlying type. Laziness is one awesome feature in languages that support it like JavaScript. If you have played with JavaScript currying, you know what I am talking about. Iterators are lazy heads so I might thing its messed up, but messed up is most of the time, cool, in JavaScript. We will look into the benefits of how being lazy can be useful. We will also see some of the powerful high level of abstractions we can achieve with iterators in ES6.

Before we move to the the for of loop, let's compare for, for in and basic iterable usage in JavaScript.
Problem: Loop over an array and sum the even values present in the array.


let sum = 0;
let arr = [10,15,20,25,30];

//////////////////////////////////////////
//Using For Loop
//////////////////////////////////////////

for(let i=0;i < arr.length;i++){
    
    if(arr[i]%2===0){
        sum += arr[i];
    }
}

console.log(sum); // 60

sum = 0; //reset value

//////////////////////////////////////////
//Using For In Loop
//////////////////////////////////////////
for(let i in arr){
    
    if(arr[i]%2===0){
        sum += arr[i];
    }
}

console.log(sum); // 60

sum = 0; //reset value

//////////////////////////////////////////
//Using Iterator
//////////////////////////////////////////

let iterator = arr.values();
let next = iterator.next();

while(!next.done){
    
    let v = next.value; // Keeping it in a new variable is not necessary

    if(v%2===0){
        sum += v;
    }

    next = iterator.next();

}

console.log(sum); // 60

for - of Loop

Before we get into for-of loop and why we should use them, lets quickly recap how for-in loops work in JavaScript. for-in loop that we saw in last example has been in JavaScript since forever and even works in IE6. for-in loop works by iterating over keys. This means, for-in can be used to iterate over arrays, as we already did, as well as an object. Using it over an object fetches all keys of the object and then we can get to values via keys.



let project = { name: "Learning ES6", version:1, blog:"anasfirdousi.com"};

for(let i in project){
    
    console.log(`Value of ${i} is ${project[i]}`);

}

// Value of name is Learning ES6
// Value of version is 1
// Value of blog is anasfirdousi.com


for-in is useful but there are scenarios in which we don't want to iterate over keys, in fact we may not care what the keys are. We only want to iterate over values without caring about keys or indexes of that colleciton. In fact, in iterables, there is no concept of keys. for-in construct can not work with iterators because it is heavily dependent on keys. Let's use the new for-of loop on the same sum of even numbers problem and compare it with the use of iterator.



let sum = 0;
let arr = [10,15,20,25,30];

//////////////////////////////////////////
//Using For Of Loop
//////////////////////////////////////////
for(let i of arr){
    
    if(i%2===0){
        sum += i;
    }
}

console.log(sum); // 60

sum = 0 ; //reset 

//The above for-of implementation is technically equivalent
//and uses the iterator behind the scenes

//////////////////////////////////////////
//Using Iterator (We already saw this example, just repeating)
//////////////////////////////////////////

let iterator = arr.values();
let next = iterator.next();

while(!next.done){
    
    let v = next.value; // Keeping it in a new variable is not necessary

    if(v%2===0){
        sum += v;
    }

    next = iterator.next();

}

console.log(sum); // 60


Behind the scenes...

Any collection/object that is iterable which has the ability to give you an iterator has a special method. It's called the [Symbol.iterator]() method.

Symbol is another new construct in ES6 but we won't be diving deep into Symbol. Keep reading to get an idea.
Let's convert our Iterator example and rewrite using [Symbol.iterator](). Whenever you call the.values() on an iterator, you are actually making a call to [Symbol.iterator](). Lets see that in action.



let sum = 0;
let arr = [10,15,20,25,30];

//////////////////////////////////////////
//Using Iterator (One again but with [Symbol.iterator]() this time!
//////////////////////////////////////////

let iterator = arr[Symbol.iterator]();
let next = iterator.next();

while(!next.done){
    
    let v = next.value; // Keeping it in a new variable is not necessary

    if(v%2===0){
        sum += v;
    }

    next = iterator.next();

}

console.log(sum); // 60


If you want to play with Symbol.iterator in your browser developer tool, you may do something like

Custom Iterables

Let's say we have a class called Order written in ES6 and I want to iterate over in instance of an Order class?

For the sake of simplicity, each item in our order will just be a number, in a real app it may be an object or something.


class Order{

    constructor(){
        this.items = [];
    }

    addItem(...items){
        this.items = this.items.concat(items);
    }

    removeItem(item){
        if(item > -1){
            this.item.splice(item,1);
        }
    }
    
}

let orderNo1 = new Order();

orderNo1.addItem(1,2,3,4,5);

console.log("List of item in the order:");

//Let's try to use for-of loop on our custom type "Order"

for(let item in orderNo1){
    console.log(item);
}


Classes in ES6 are pretty straight forward and if you want to learn about them you can checkout the Mozilla Documentation for it. We will not be going deep into classes in ES6 in this post.
The above for-of loop won't work because for a data type to work with for-of, we need to implement a [Symbol.iterator]() property on our data type "Order". The [Symbol.iterator]() has to return an object with a well defined next() method which can be called over and over again. That next() method when called has to return result of shape { value:<any value> , done:<true/false> } You can do this in 2 ways,

  • A: Implement the iterator within the class itself. This can be a quick solution but not reusable. You will end up implementing similar iterators all over the places and 100 difference classes. If you have a very few classes, you can still do it. The code may look like:
    
    class Order{
    
        constructor(){
            . . .
        }
    
        addItem(...items){
            . . .
        }
    
        removeItem(item){
            . . .
        }
    
        [Symbol.iterator](){
    
            next(){
                // Implement custom iterator logic here
            }
    
        }
        
    }
    
    
  • B: Delegate the iterator logic to a custom iterator class which can be reused by multiple classes who want to become iterable. For our Order class, we will write a custom iterator class called ArrayIterator and return that from [Symbol.iterator](). Here is the listing, check out:

    
    
    class Order{
    
        constructor(){
            this.items = [];
        }
    
        addItem(...items){
            this.items = this.items.concat(items);
        }
    
        removeItem(item){
            if(item > -1){
                this.item.splice(item,1);
            }
        }
        
    
    [Symbol.iterator](){ return new ArrayIterator(this.items); }
    } class ArrayIterator { constructor(array){ this.array = array; this.index = 0; } next(){ var result = { value:undefined, done:false } if(this.index< this.array.length){ result.value = this.array[this.index]; this.index +=1; }else{ result.done = true; } return result; } } let orderNo1 = new Order() orderNo1.addItem(1,2,3,4,5); console.log("List of item in the order:"); for(let item of orderNo1){ console.log(item); } //List of item in the order: //1 //2 //3 //4 //5

Check for bold pieces in the code
One more thing, we will rewrite the above Order class example using generators and for-of loop right at the end. You may love it if you read till the end!
Take it easy! Let's grock it... What exactly have we done here?
  • We have made our Order type an iterable
  • We have given a meaning to for-of ing any instance of type Order. What's the meaning of looping over a instance of type Order, what should happen when someone iterates over. Before ES6, the only thing we had was for-in loop which would iterate over keys of an object but we could not give a custom meaning to traversing over custom types. With ES6 iterator, YES we can!
  • Question:What can we do in next() method?
    Answer: Literally anything! Anything you like! You can do whatever and then return a tiger(🐯) out of it , no problems! The only thing it needs to make sure that it returns an object of shape { value:<soma value>, done:<true/false> } In real apps, you may have a requirement to send items back starting from the last element. You can do that login in next(). There can be a scenario where you only want to return elements which are divisible by particular number, you can have that logic in next(). Literally any logic!
  • Our next() implementation is simple, sober and innocent. We just iterate over the array passed to "ArrayIterator" and check if it has more elements. That's all innocent next()s do.

Generators

We generate an Iterator by calling .values() on our iterables. There is another easier way. A generator function is a function that generates an iterator. It requires very less code but brings in some new keywords and syntax. Let's look at a definition and usage of generator in ES6.



    let letters = function*(){

        yield 'a';
        yield 'b';
        yield 'c';
        yield 'd';

    };

    let iterator = letters(); // Get an iterator out of a generator

    console.log(iterator.next()); // Object {value: "a", done: false}
    console.log(iterator.next()); // Object {value: "a", done: false}
    console.log(iterator.next()); // Object {value: "a", done: false}
    console.log(iterator.next()); // Object {value: "a", done: false}
    console.log(iterator.next()); // Object {value: undefined, done: true}


Things to notice...

  • Notice the star/asterisk used with the function keyword.
  • Instead of using a return keyword to return a value, we use the yield keyword to return multiple values over time i.e. whenever we call .next();
  • Did you notice how easily we produced a iterator out of a generator function, just by calling the generator function.
  • Generator function can be thought of as a factory for iterators.
  • What's more important than anything here is whenever you make a call to next() on the iterator, produced by an ES6 generator, it returns the value on the first yield and yields/returns the call of execution to the caller. The value is returned to the caller immediately and the caller can do whatever it needs to do with that value, perform manipulations, do logging, do whatever, and then when the caller asks iterator for the next value, execution will return to the generator function just where it left off and if you call a next on it, it will execute and run the second yield value and then returns execution to the caller once again. The generator function always remembers where it left off and always returns the returns the next 'yield' value. Its a magic! Isn't it. How it happens? We will get to that in a bit.
  • Rephrasing the last point, generator function suspends its execution when it gets to a yield statement and resuming later when someone tries to get the next value.
Ready for another example on generators? Let's do it!

Problem statement: Calculate the sum of first 100 numbers.



    let numbers = function*(){
        for(let i=0; i<=100;i++){
            yield i;
        }
    }

    let sum = 0;
    let iterator = numbers();
    let next = iterator.next();

    while(!next.done){
        sum += next.value;
        next = iterator.next();

    }

    console.log(sum); //5050

    };

You can also pass values to a generator function like you'd do to any other function.

Problem statement: Calculate the sum of a range of numbers when start and end of the range is passed.



    let numbers = function*(start,end){
        for(let i=start; i<=end;i++){
            yield i;
        }
    }

    let sum = 0;
    let iterator = numbers(1,100);
    let next = iterator.next();

    while(!next.done){
        sum += next.value;
        next = iterator.next();

    }

    console.log(sum); //5050

    };

As we talked earlier, we can get rid of the step of producing an iterator separately and then doing .next() to generate a value, by simply using the for-of loop which does all of that for us automagically. Example?



    let numbers = function*(start,end){
        for(let i=start; i<=end;i++){
            yield i;
        }
    }

    let sum = 0;
    for(let n of numbers(1,100)){
        sum += n;
    }
    
    console.log(sum); //5050

    };

Do you remember our Order class example we discussed where we also wrote an ArrayIterator? As was promised above, let's rewrite that example using Generators, you don't actually have to write iterator class (like ArrayIterator) that we wrote for most of the common scenarios.



class Order{

    constructor(){
        this.items = [];
    }

    addItem(...items){
        this.items = this.items.concat(items);
    }

    removeItem(item){
        if(item > -1){
            this.item.splice(item,1);
        }
    }
    
*[Symbol.iterator](){ //Converted this to a generator function for(let n of this.items){ yield n; } }
} let orderNo1 = new Order() orderNo1.addItem(1,2,3,4,5); console.log("List of item in the order:"); for(let item of orderNo1){ console.log(item); } //List of item in the order: //1 //2 //3 //4 //5

How cool is that! We got rid of a lot of lines of code there. I would suggest using generators over writing a custom Iterator class unless there is a complex iterator logic that you want to implement.

Benefits of Using Iterators/Generators (Take aways)

  • Iterators and Generators allow you to write composable functions which can help you build independent composable function pipelines. Boi! you gotta love that if you are a functional programming lover! We will look at a quick example right next.
  • Generators, if you try to look at them closely by now, they posses another nice feature of JavaScript language and functional programming called lazy evaluation.

You may have used other utility libraries like lodash and underscore where you get a lot of functions implemented out-of-the-box. I would like you to think of mapping functions that pass values through. Isn't that the essence of pure functions! Let's implement three such functions, filter,map and take

Let's re-write the order example with lots of items in the cart.


class Order{

    constructor(){
        this.items = [];
    }

    addItem(...items){
        this.items = this.items.concat(items);
    }

    removeItem(item){
        if(item > -1){
            this.item.splice(item,1);
        }
    }
    
     *[Symbol.iterator](){   //Converted this to a generator function
          
          for(let n of this.items){
              yield n;
          }

    }

 }

//Naive implementation of MAP
let map = function*(items,mapper){ 
      for(let item of items){
        yield mapper(item);
      }
}

//Filter
let filter = function*(items ,predicate){
    
    for(let item of items){
      if(predicate(item)){
          yield item;
      } 
    }
}

//Naive implementation of TAKE
let take = function*(items, number){
    let count = 0;
    if(number < 1) return;
    for(let item of items){

      yield item;
      count = count + 1;

      if(count >= number){
        return;
      }

    }
 }


let orderNo1 = new Order()
orderNo1.addItem(10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80);

console.log("List of item in the order:");
  
//ES6 style function consts  

const square = n => n*n;
const even = n => n%2===0;

//Check below how can can create independent composable pipelines using generators

for(let item of filter(map(take(orderNo1, 2),square),even) ){ console.log(item); } //List of item in the order: //100 //400

Wait, wait, wait... lem me explain...

If you understood what we just did, awesome!!! I think you are good to go on ES6 generators! However if you are lil confused about the above example, not a problem. Let me explain.

There are 3 major things to consider

  • We wrote 3 composable functions using generators, map, filter and take. I am sure not the best implementation but that's not the point here. We are trying to write composable functions using generator's lazy evaluation power.
  • ES6 style functional consts. If you would like to read about const in detail, you may want to look at a previous post on ES6 by clicking here.
  • Last and probably the most coolest thing we did with generators is on our for loop.

    
      for(let item of filter(map(take(orderNo1, 2),square),even) ){   
          console.log(item); 
      }
    
    

    The best way to understand this is to think of independent pipeline from right to the left.



    I would suggest you compose these pipelines in your head without thinking of the "type" of data source (collection). A collection can be anything, an array of numbers as in our example, an array of objects, a event stream of mouse clicks or drags or any type of data stream connected or disconnected. And then you apply your composable functional pipeline on that data source as the data arrives. This concept can help you understand how to think more in terms on functional and reactive programming paradigms.

    In the above example, we take the incoming data one by one. So suppose you are on item x. Pass 'x' through a take function. Take is trickier to implement. We need to memoize and remember how many items we have already passed through. Since we ask it to pass only "2" items, it won't pass more than 2 items and will stop the pipeline. So once it picks up the first number say 'x', it pushed 'x' to our map function. We pass x*x to our map generator function as a mapper. So item 'x' after passing through map becomes x*x. the result of x*x is then given to the filter function which checks if 'x' is even, if it is, it passes it through and push the value to item in our for-of loop.

    The order in which you apply functions in a pipeline matters. It will eventually give you the same result because composable pipelines obey set theory but changing order can optimize your composable pipelines. Hint:What if we interchange positions of our "take" and "filter" function. Put a console in each of these functions and try that!

What's coming in next post...

I initially thought I should discuss "Comprehensions" in ES6 within this post but in the spirit of love for people with short attention span (like me!), I am deferring that to my next post. Consider this post and any knowledge which you may have gained here as a pre-req to learning Comprehensions in ES6. I hope you have learned something out of this post and will apply it in your projects.

Conclusion

Feel free to leave a comment, question, suggestion and corrections. Until next time, Happy learning!