April 9, 2013

Using AngularJS at Localytics

Share

This is the story of how we rewrote our Backbone-powered web application in AngularJS, using nearly half the number of lines of code.  (It is a love story.)

Our Time With Backbone

In the beginning, there was spaghetti code. Then out of the chaos came Backbone, and it was good. Our data lived in Models which lived in Collections which were observed by Views, and our application grew. But with growth came complexity, and as we began to nest our Views in deeper layers, evil Zombie Views plagued us, and we despaired.

Backbone promised a simplicity that eluded us in our real-world application:

“In a finished Backbone app, you don’t have to write the glue code that looks into the DOM to find an element with a specific id, and update the HTML manually — when the model changes, the views simply update themselves.”

We found this statement misleading.

In Backbone, View.render is a no-op, so the above would perhaps be better written: “when the model changes, you are responsible for binding your view to the model’s change event and writing a render method that can be called each time this event fires, and make sure to unbind it when you’re done.” For us, this was easier said than done.

First, to keep our views manageable, we followed thoughtbot’s example and wrote composite views made up of smaller views responsible for discrete parts of the page. Parent views had to keep track of child views and make sure to clean up after them. And with nested views, it became less clear what the render method in each view was supposed to do.  Should it render itself and then recreate and rerender all child views?  Should it reuse existing child views and just tell them all to rerender? We ended up investing many lines of code and development time hacking together composite view library classes to help manage these cases, not to mention endless hours spent hunting down zombie views that weren’t being cleaned up correctly.

Second, our app was heavy on user input, and we never found a good way to keep our form inputs, models, and views in sync. First, we began by writing glue code that looked into the DOM to find an element with a specific id and grab the data from that element on keypress. That caused the model to change, so sure enough, our observant view rerendered, causing the input to lose focus. So then instead of simply rerendering the entire template, we wrote more glue code that looked into the DOM to find an element with a specific id and changed the value of that element to reflect our model. Now we had double the amount of awful DOM manipulation code in our views, so we turned to a little two-way data-binding library called Stickit. The plot thickens.

The tale of our journey down the path of Stickit is an epic in itself, but long story short, it only made things worse. Stickit’s philosophy is to “clean up your templates” (make them less declarative) by moving your data-binding logic to your view. So now we had a whole library whose purpose in life was to litter our views with the names of more DOM elements. Our zombie views grew fatter.

But the straw that broke the camel’s back was Chosen, the jQuery plugin that turns ordinary <select> elements into pretty searchable dropdowns, and an essential part of our UI. It doesn’t work with Stickit and it doesn’t work with Backbone views that haven’t yet been inserted into the DOM. But we persisted, writing tomes of code to coerce these three strangers into playing together. So when it came time for a UI revamp of our app marketing platform, we realized we were going to have to rewrite enough of our existing code that we jumped at the chance to try something new.

Into Angular Land

We were ready for a framework that offered more out of the box, and any moral objection we might have had to soiling our templates with non-standard HTML had been completely beaten out of us at this point, so AngularJS’s declarative two-way data-binding was starting to sound real good. When we found AngularUI, complete with datepicker and a Chosen lookalike ready to go, we were sold.

It was, to be sure, mind bending at first. Without our familiar friends, Model and Collection, we weren’t even sure how we were supposed to model our data, a question on which AngularJS is conspicuously unopinionated. We also weren’t sure how to organize our code, and the Angular documentation wasn’t very helpful here, as it’s riddled with sample code which it then advises you not to follow in a real application. But a few days in, it was evident that we were writing so little code in Angular anyway that it didn’t really matter if our organization wasn’t perfect.

We finished the project in two weeks and in about half the number of lines of code, including HTML markup, as our old Backbone version.

Localytics AMP UI - Backbone vs Angular

Data-binding worked like magic. We wrote zero lines of DOM manipulation code, and spent zero minutes tracking down memory leaks or unpredictable event binding behavior.

As an unexpected bonus, Angular’s module API and dependency injection system also turned out to be way cooler than we imagined. In Backbone we had struggled with a hand-rolled module loader to organize our code, and it took no small amount of discipline to keep passing all the variables needed into our views (which were nested several layers deep) instead of throwing our hands in the air and just using the module loader as a place to stick global variables. Now Angular takes care of all of this for us, managing our services and providing instant access to them wherever we require.

As it turned out, the select2 dropdown widget that comes with AngularUI didn’t quite meet our needs. The author acknowledges that the plugin is slow with large datasets, and we wanted something that would work with ngOptions. So we wrote our own directive for Chosen. The directive works on any <select> element, and plays well with ngModel and ngOptions:

 <select multiple chosen ng-model="state" ng-options="s for s in states"> </select> 

Lessons Learned

To close, here are a few things we learned from the transition, and a few questions we’re still working on answering:

Directives are hard. Working with isolate scope and transclusion is tough, and Angular’s documentation on the subject doesn’t make it easier. We found that before jumping into writing an ambitious directive, it’s best to start by not writing a directive — just using normal templates and controllers — and then roll that code into a directive once you really figure out what your requirements are, or once you start repeating yourself.

It’s also best to write smaller, less obtrusive directives that play well with others. For example, we went through a few iterations on the Chosen directive before realizing that it was best to keep it simple and define it as an attribute on a standard <select> element, instead of creating a <chosen> element that would render its own template.  By keeping it lightweight and retaining access to the <select> element outside of the directive, we can still add other directives on top of it, like ngModel, ngOptions, ngChange, or any of the directives that work with validation.

Angular documentation needs some love. But we were able to figure it out anyway, and it turns out there isn’t as much to learn as we’d initially feared to get started. The developer’s guides to directives and forms are now our two best friends.

Form validation is great but requires some thought. Our application helps customers create targeted in-app messaging campaigns through what’s essentially multi-page form wizard, so dealing with form and model validation was tricky. Angular’s built-in validation directives like ngRequired helped us eliminate a huge chunk tedious validation code that had previously lived on our Backbone models, but we still had some model-related business logic that couldn’t be defined declaratively, and which needed to be persisted if the user left our app and came back later without stepping through each page of the form again. Integrating model and form validation was tricky, and our implementation probably needs some rethinking. We were also surprised to find that Angular doesn’t seem to support ‘required’ validation on groups of radio buttons, and had to resort to adding a hidden input element with a required attribute to validate that a user had selected an option.

Declarative is good. It’s funny, but we’ve seen interactive web apps with HTML and Javascript come full circle.  We used to write things like <a href=”#” onclick=”doSomething()”>. Then we realized it was bad to couple presentation and behavior, so we made our Javascript unobtrusive, keeping our templates clean. But now we’re back at it again, writing <a href=”” ng-click=”doSomething()”>. Have we learned nothing?

My experience with Backbone has led me to conclude that the ethos of “keeping one’s templates clean” is an antipattern that creates brittle connections between Javascript code and DOM structure. Web development is messy business, and the ugly code that brings HTML to life has to live somewhere. By abstracting it out of our templates, it just clogs up our views with hardcoded references to HTML elements whose names, classes, and structure might change — and which should be allowed to change without breaking behavior and needed a corresponding change in a separate view class.

Philosophy aside, Angular helped us get our job done with a lot less code, leaving us more time to do the things we love, like drink beer frolic in the sun write more code. Plus the t-shirts are pretty cool.

Posted in Company & Products
Share this
Share
  • jessegavin

    Awesome write up. While I haven’t used Backbone before, I recently switched from KnockoutJS to AngularJS and have experienced a great deal of benefit from it, especially in terms of the amount of code I don’t have to write.

  • http://twitter.com/whomba whomba

    Great article. I have had similar experience diving in to Angular from scratch. I had an almost identical experience with directives, Keep it simple, and they are great, make it complicated and you will be in pain. I was also quite impressed with the amount of code I DIDN’T have to write. I would say I saved around 30% of LOC on my final project as well as 15-20% of the time it took to build (probably now that I’m much more familiar with it, that would be 20 – 25%)

    One thing I disliked is that plugins I had and work beautifully with jQuery take some massaging to get with this, or is strongly frowned upon, as unless something is in a directive, it shouldn’t touch the DOM. So as a result I spent some time re-inventing the wheel on some items.

    I do agree that the documentation needs some love. There were many times I just was left clueless. I got around this by diving in to their code (which helped the most) and posting on their [fairly active] google group.

    -Andrew

  • Jeremy

    Thanks for the comments about keeping directives simple. I was close to moving beyond angular and taking up another framework due to my frustration with directives, but I was trying to learn how to write them by starting with a very complicated one. I do love the elegance of the library, it just seems like there is a hurdle to get past for people coming from non-functional programming backgronds.

  • http://www.johncblandii.com John C. Bland II

    I would say http://egghead.io/ is a prerequisite for doing directive work. :)

    Great post!

  • dwelch2344

    I feel ya! I also went through the same directive over-engineering cycle (as did a few other devs I know) but eventually figured out taking the simplistic approach. Now the general rule seems to be “if it’s painful, you’re probably doing it wrong”

    And yeah, I agree that there can be some reinventing the wheel when it comes to working with already existing components, but it really makes sense if you understand “why” Angular is awesome. Separate those concerns :D

  • Capaj

    I myself consider Knockout even more magical than AngularJS in terms of data-binding. Angular is much more suited for big single page apps, Knockout is just about those magical bindings…

  • http://www.johncblandii.com John C. Bland II

    I beg to differ. Angular works for small widgets as well.

  • http://twitter.com/eranation Eran Medan

    Great article,
    One question, if you had a legacy app, that you needed to refactor (not rewrite) with a lot of jQuery document ready spaghetti, and needed to do so gradually (e.g. one piece at a time other than doing the entire app) would you still move to Angular over Backbone? I thought that Angular is great for new projects, but it’s kind of “all or nothing” e.g. the Angular way or the highway, did I get the wrong impression?
    Is there basically no future for Backbone if it doesn’t evolve?

  • http://www.johncblandii.com John C. Bland II

    You can mix it in. ng-app can go on any element in your DOM (top html or individual div). You can also place it on the top html element but only apply controllers to specific parts of your site.

    If the auto-instantiation of the app isn’t desired, you can manually do so within your “document ready spaghetti” by manually calling angular.bootstrap (http://docs.angularjs.org/api/angular.bootstrap).

  • Capaj

    I beg to agree. Yes it works for small widgets, but it is not as magical-with Angular if you change anything programatically in your scope and forget to do scope.apply(), the view won’t update. It is not magic. Knockout you have to get used to ko.observable magic, but once you do, everything updates literally magically. I just think it is more magical. Though I build all my projects on Angular now, because Angular is definately more practical.

  • http://www.johncblandii.com John C. Bland II

    No, you shouldn’t need to use $scope.$apply() after any updates if you make them from within Angular/the scope. If you find yourself having to do a lot of $scope.$apply calls, something is amiss in your setup.

    I haven’t used Knockout so can’t say. For clarity, I differ on the part of Angular being “much more suited for big single page apps” and not comparing to Knockout.

  • nateabele

    If you’re having trouble with directives and jQuery, check out the AngularUI project. They have some good tips/templates for handling that interaction.

    With regard to jQuery plugins being frowned upon, it’s because Angular’s design completely obviates the need for the vast majority of them.

  • Capaj

    I know when to use $scope.$apply() thank you very much. I write regularly lot of directives and there are tons of cases where Angular will not detect any event and will not automatically trigger dirty checking.

    And for absolute clarity-I was talking in context of comparing the two when I wrote “much more suited for big single page apps”. So I think we do not disagree at all :D Angular is great even for even the smallest pieces of your web app, though it’s size predestinates it for larger apps, where it’s 0.5 MB(non minified) won’t do any harm.

  • colynb

    most of what happens inside scope.apply happens for you internal to angular so you shouldn’t have to be doing that. Nice article on the subject http://jimhoskins.com/2012/12/17/angularjs-and-apply.html

  • colynb

    I replied before I saw this. So you already know what I’m talking about. Sorry.

  • http://twitter.com/delambro Matt DeLambo

    Stickit author, here. I’m interested on hearing how your “zombie views grew fatter” after using stickit. Stickit takes good care in removing model and view bindings automatically on view.remove() and manually through view.unstickit().

    Also, I’m guessing you were working with a version of stickit before 0.6.3, because the new handlers interface lets you build a global handler for plugins like Chosen and much more.

    Example of how to create a global binding for a plugin like Chosen.

    More on handlers.

  • Joel Rosen

    Stickit contributor here :)

    https://github.com/NYTimes/backbone.stickit/issues/66 in particular created an explosion of zombie views, but yes, it’s fixed now.

    Our case with Chosen was more complicated because we were populating the select collection with the results of a remote query, and because we were using a forked version of Chosen that allowed the user to enter custom options if they weren’t in the collection. It was not easy to get these to work with stickit because with updateView turned on, stickit would clobber the contents of the select element, and with updateView turned off, stickit wouldn’t recognize the values in the select element because it required a stickit-bind-val stored using $.data().

    Anyway I’m pretty sure we were never using stickit quite the way it was intended to be used, since we preferred rerendering views instead of having stickit update the DOM. And by using stickit, we weren’t really using Backbone the way it was intended to be used either, since they discourage use of two-way data-binding libraries. It was just not a good match for us.

  • http://twitter.com/delambro Matt DeLambo

    Right, data binding is not needed for every app.

    The way you were using stickit was unusual, but we still fixed that rare use case where someone might want to re-render a data-bound view, and this was a brief problem in between releases. Normal users never came across that problem, and in normal use, stickit is very good about cleaning up after itself. So even after your rare use case and short-lived problem was patched, do you still think stickit causes zombie views?

    Blogging that it isn’t a good match for your project is different than claiming that it causes memory problems…

  • http://pagesofinterest.net/ Michael Robinson

    Great article, and I agree with you that directives are hard. I find them beautiful though.

    A colleague discovered this: http://ngmodules.org/ which will hopefully grow into a hub for 3rd party Angular code.

  • http://www.johncblandii.com John C. Bland II

    I’m not questioning your skills Capaj just noting incessant use of scope.$apply is misleading and unnecessary. Manual use is needed for specific scenarios, I’m not denying that, but not basic binding.

    Cool on clarity.

  • Nate Tuganov

    Thanks for this post. I’m curious do you have any performance comparison between backbone and angular on the client side?

  • http://liveditor.com/ Edwin Yip | LIVEditor Dev

    I haven’t had a chance to use AngularJS yet but I really like it, your real life experience will help me to make such decision in the future, if I would use it, I’ll add code-complete support for AngularJS in my LIVEditor project (http://liveditor.com = text editor + browser + Firebug).

  • Steve Castle

    If you guys have worked with ember.js as well I’d love to hear your opinion on it compared to your experience with Angular and Backbone.

  • iansltx

    Well-done article!

    Re: Declarative Is Good, it seems like the split-up of presentation and behavior school happened when there was more presentation and less behavior, such that the idea of “this” being scoped to an individual element would violate DRY. Now that we’ve got CSS3 et al, JS’s job is to do heavier lifting, working on individual elements rather than style-oriented classes. So you get into OOP, where “this” is a really helpful construct.

    At least, things seem to be going that way. Take the above with a grain of salt, as I’m a server-side guy by default, such that the largest JS I’ve written is < 200 lines. That'll change soon though…AngularJS, here I come!

  • Peter Drinnan

    Less code was a big reason for my selection of Angular for one of our projects. Ember looks really good too but I decided to go with Angular because it has a smaller footprint and looked easier to learn. Still thinking Ember will be good for some bigger projects.

  • Rafael Jesus

    Angular seems to be the new jsf for Java, and when ya need to use pure JavaScript, ya don´t remember how to use it anymore

  • http://www.johncblandii.com John C. Bland II

    That’s how I see jQuery. People don’t know how to traverse the dom w/out $.

  • Rafael Jesus

    yeap

  • auser

    For directives, try checking out: http://www.ng-newsletter.com/posts/directives.html

  • Brad Gearon

    Great story. You crack me up also.
    I used this in part to help formulate ideas expressed here: http://bradgearon.github.io/cohesive/
    Please feel free to read and provide feedback. Its not finished though (I describe the problem – not the solution – but I’m still working on it).

  • Brad Gearon

    So I’ve found myself calling $scope.$apply erroneously. I am always learning, but I’m pretty sure I have a working say – principle – I follow to determine if I need to: if its a DOM event (normally one I bind to in my directive) I need to call apply. If its a property change – say from $watch on an attribute – apply will be automatic (and if I do call it – well I’m thankful for stack overflow detection – much simpler than when I accidently render the parent template recursively and have to kill the tab or browser).

  • Brad Gearon

    Sorry to post this twice, but I see where you are coming from. For the “unobtrusive” jquery plugins I’m pretty sure this is why:

    http://bradgearon.github.io/cohesive/

  • Brad Gearon

    Shouldn’t be in the first place

  • Ted Jenkins

    Cool stuff. I too am migrating a backbone app over to angular. How did you do routing during the conversion process? Did you have angular handle your routes, or backbone?