Building REST services, rambling …

With the recent changes in the web development like SPA and change in device platforms, the backend changed drastically.  Backend no longer just serving up pages, they are now moved to highly scalable, available and responsive service providers. Over the past year or so, I was purely focused on the front end development. I did not touch much of the backend since we had very talented people in the back end. But over the last couple of months, some of the changes happened in the backend made me look at the backend architecture. Similar to building highly testable and manageable front end, back end also should be able to sustain changes with out breaking the front end. That means, when you build services, you need to build them in such a way, they are self serving and highly discoverable services which can adopt to changes without breaking the clients.

Our back end is build with Web API. It is easy to put together simple hello world or student REST services. But when building enterprise web services, you need to put some time there to come up with what are the resources are you going to expose and how are you going to expose. For those who have been developing services using SOAP protocol, this is paradigm shift. So if you are going to build REST services, don’t just convert the SOAP to REST rather think about what are you trying expose.

I remember reading somewhere, building REST URL, resources and query parameters is 80% art and 20% science. I totally agree to that. When I sat in discussions initially, I was thinking it was pure waste of time. It is not a rocket science, but later I found out, building rest service end point is thing of beauty. Take time and look at the resources you are going to expose and why? Ask the question, are they really the resources clients want? Just because we have a resource in a table doesn’t mean we need to expose them as a resource. When you are exposing the resource, it should be meaningful to the end user and easy to use.

I am not going to explain how to build REST services. Vinay Shani wrote one of the best blogs on the REST services. Here are some of the major things I went through when I was consuming REST services and moved to building services.

  • Cache: Build your rest services in such a way, it leverages the caches. Poorly designed resources end up not using caches. Especially when you are building highly responsive services, cache is an important piece of the puzzle.
  • Versioning: There are three different ways one could version the services. I personally do not like versioning in the URL but on the other hand, we do not have mature technology to enable discovery either. I remember reading somewhere, if you start out with versioning, you are already admitting your failure. I loved this blog on versioning. Anyway you do it, you are doing versioning wrong :)
  • Query string/Resource hierarchy: Take care on how you want to set this up. This goes hand in hand with Cache. I prefer resource hierarchy than query string. I prefer to use query string only in how I want to the representation to be and filtering my results.
  • Documentation/Discovery: You could create an awesome service, like Taj Mahal and put in the middle of a forest where no one can find it, then there is no use. Also if you build mind blowing machine and if it is not easy to use what is the point. Build great API documentation, use tools like swagger or some other forms. Provide great on boarding story and excellent means for users to report problems and concerns like Forums/Discussion groups.

There are ton of resources available on the web. Before you start wiping up REST services, change you mind set, read what people went through ask questions and be smart in building resources. Like Troy said in his blog, once people start consuming, they are very important part of the equations so changes have to be handled very carefully.  We should not create distrubtive services that clients do not want to use. Only way you services be successful is if you clients consuming it and happy using it.

Building your movie website with Movies DB in angularJS

I am a big fan of movies, for some time, I have been tinkering with building a website to see what is new on movies and TV. Recently I came across Movies DB website. The website by itself is wealth of information but the best part is, if you would sign up with them then you can use their rest services. You need to read the legal documents to see if it fits your bill also they throttle the requests coming in as well. I set out to write my own service so that I can use this service in my couple of other projects I am working on. Well, even that is already solved for me. Head out to lub-tmdb at GitHub, they have done most of the work for you already. So it is just a matter of hooking it into your project.

They have an excellent example project in their on how to use it. The example project is self contained, so I thought I add couple of lines to show, how to add into your existing project.

1. Download the min file from their build directory and put it in directory of your choosing. In my test code example, I put it in the root directory.

2. Add the script file to your index file, the real important thing is, it has to be added after your script with ‘angular.module’ definition. You would want to do the same if you build your own service.

<script src=”https://ajax.googleapis.com/ajax/libs/angularjs/1.2.18/angular.min.js”></script>
<script src=”app.js”></script>
<script src=”lub-tmdb.0.1.1.js”></script>

As you can see ‘lub-tmdb.0.1.1.js came adter my main app.js file.

3. I followed their example and created a simple search for a movie like the following in the index.html

<form ng-submit=”exec(‘search’, ‘movie’, query)”>
     <input type=”text” ng-model=”query”>
     <button type=”submit”>Search Movie</button>
</form>

<div>{{searchResults}}</div>

4. In the controller, again based on their example, I called the service as follows;

var app = angular.module(“main”, [‘lub-tmdb-api’])
     .value(‘lubTmdbApiKey’, ”);

var mainCtrl = function ($scope, lubTmdbApi) {

    var suc = function(result){
         $scope.searchResults = result.data;
     };

    var err = function (results) {
         $scope.searchResults = results;
     }

    $scope.exec = function (type, method, query) {
         lubTmdbApi[type][method]({
             query: query
         }).then(suc, err);
     }
};

app.controllers(“mainCtrl”, mainCtrl);

The important points in here are

  • Second line the above script, should have your api key that you got from the website.
  • Don’t convert it to JSON object in this line; $scope.searchResults = result.data;

I personally really enjoyed using both the angular service @ lub-tmdb and also the Movies DB. Both are solid and made my life little easier. Thanks a lot for both the services.

Stumbling through express & using middleware

I took a break from Meteorjs and went back to nodejs to understand what is special about MEAN (Mangodb, Express, Angularjs and Nodejs). While I was reading through MEAN I realized I did not do much with node or express for that matter. More I read about express, I realized how easy it is to get going on it. So I started to read couple of blogs and trying out simple todo project to understand express.

Here are couple of things I ran into so far;

  1. It seems you need to install express, connect and jade in the local project directory. For some reason, even though I installed it with –g option, it was not picking it up. There must be something I am doing wrong here but for now I made it to work with local install.
  2. I spend some time figuring this out, make sure the routes are properly defined. I had the following

app.get(‘/’, ‘home.index’)

as you can guess, ‘home.index’ is routes to JavaScript code which shouldn’t be in quotes.

3. This is important, if you are trying to follow some blogs, you will run into problems as express evolved and some of the blogs are outdated (so will this). But as of 0.10 version, middleware portions of the express were removed from it and it is now available under connect. So if you want to use middle ware, in my case, I want to read and parse the body part, I need body parser which is now part of connect npm package. So to access something like the following in my controllers

name: req.body.name

I need to do the following

  • npm install connect
  • in the require, add

middleware = require(‘connect’)

next you need to add body parser to the express to use

app.use(middleware.bodyParser());

One side note from ‘Good parts of Javascript’: ‘Make a habit to have curly braces on the right of the statement instead of the next line’. Fortunatly coming from C#, I always practiced that so I am in good shape Smile

Be mindful of the method names case sensitivity

Today I was trying to rebuild my sample application once again to understand the meteor framework once again (Yes, I like to repeat multiple times) and ran into run time error.

It turned out I had ‘Collection’ as ‘collection’. Collection has to start with capital C. I used webstorm to autocomplete the word and it never occurred to me that could be the problem.

Anyway, this is just note to self so I do not run into the same problem again.

Adding meteor package in Windows

I have been playing with Meteor on my mac after initial hiccups in Windows. Now I got a hang on it, I thought I give it a try again in Windows and turned out, it is was not that hard.

You need to start out at http://win.meteor.com/ it has different ways to get started in Windows.

I was able to get my sample app up and running in no time. After some simple Hello Worlds, when I started to dig in deep to Meteor, I hit the wall when trying to add a package that was not available through Meteor. In mac you can use ‘mrt’ and everything was peachy. It turned out it was not that difficult to install the packages in windows either. It takes couple of extra steps but other than that it is straight forward. There are an excellent set of instructions available here at https://www.discovermeteor.com/blog/using-meteor-and-atmopshere-on-windows/ and also at http://stackoverflow.com/questions/20949780/how-to-install-meteorite-for-windows

The simple idea is to create folder called packages under project root and clone the package project that you want to use from git. In my case I was trying to use iron-router. Here are the steps I did, on DOS box, go to project/packages folder

git clone https://github.com/tmeasday/meteor-router.git

This should create the meteor-router folder, rename it to iron-router and then clone the three dependencies you need to build iron-router. Stay on packages folder and run following git clone commands. This might change as Meteor matures.

git clone https://github.com/tmeasday/meteor-page-js-ie-support.git

git clone https://github.com/tmeasday/meteor-HTML5-History-API.git

git clone https://github.com/EventedMind/blaze-layout.git

After it downloads remove the ‘meteor’ from the folder names. Also make sure all the files made it to the sub folders. If you have problem getting files down, change to the package folder where you need to get missing files and run the following command

git submodule update –init

I had to run the update in both page-js-ie-support folder and HTML5-History-API and then rerun the git clone again.

Once you have all the files are in place. In DOS box go back the project root folder and install the iron-router.

meteor add iron-router

With the above steps, I was able to install and run the router like a charm in Windows.

Good luck.

Using jQuery UI controls in MeteorJS

If you want to work with jQuery controls or third party controls for that matter, you can use them with single caveat. You need to have a package available for it to use in the Meteor application. I found an excellent blog on this here. So I am not going to repeat the whole thing here. But it was missing one important thing, you need to install the jQuery UI package first.

mrt add jQuery-UI

or

mrt add jquery-ui-bootstrap

Also if you want to create a package, here is great links

https://atmospherejs.com/docs/publishing

http://stackoverflow.com/questions/19955665/how-to-create-meteor-package

User management in Meteor is so easy

I was pleasantly surprised how Meteor makes the developers life so easy. The official documentation for data security is available at http://docs.meteor.com/#dataandsecurity

When building web applications you might want to authorize users before using the application to the fullest extend. By default when you create the meteor application, any changes that happens at client and server will be seen by everyone else. By default, auto publish is enabled. When you go to production, you might not want everyone to see everyone’s changes. You want to authenticate and authorize users. This is standard task in any web applications. You end up coding once and then maintaining the code. Most of the time you create your own user tables and then perform email verification etc., As you can imagine, this is repeated task when developing new application and has clear pattern. Meteor framework team understood this pattern and provided you with easy to use solution. So in my code, with just 5 or 6 line of code, now I am able to provide login using,

  1. Creating user name and password.
  2. Creating user id with email and password.
  3. Twitter
  4. Facebook
  5. Google.

This makes developer life so easy. In the HTML all you have to add the new template provided by Meteor like the following

<div style="float: right; margin-right: 20px;"> {{> loginButtons}} </div>

Now this alone will not get you going, you need to add the meteor packages for making it work. So in the project terminal, add the following based on your requirement

By default you need to add following for option (1) and (2)

meteor add accounts-base
meteor add accounts-password
meteor add email
meteor add accounts-ui

Once you add all the above four packages, you will get Sign in option show up in the main page like the following

image

Now to make it work, you need to say, do you want to create users by just user id and password or email and password. This configuration need to happen in account ui configuration. You put the following code in client side javascript code

Accounts.ui.config({
   passwordSignupFields: 'USERNAME_AND_OPTIONAL_EMAIL'
});

Lets suppose you did add twitter/google/facebook by adding those packages in the console with following command

meteor add accounts-twitter
meteor add accounts-google
meteor add accounts-facebook

then your sign in screen will look like the following

image

Now, if you you want to authenticate using facebook or twitter, you need to register you all application with those identity providers before you can allow the users to authenticate.

But overall, with few lines of code, I got my user authentication completed and now I can focus on my actual application development.