Many articles describe the interaction between Node.js and Elasticsearch, but they often do not clearly explain how this interaction was achieved. To fill this gap, this article describes test-driven, step-by-step development of a simple RESTful API into an Elasticsearch in Node.js.

My main intention is to show Node.js developers how a RESTful API might be written using a TDD approach. Applying TDD practices makes the process much faster and results in a less error-prone API. Moreover, the whole application architecture becomes testable and therefore simpler and cleaner.  

This article is divided into three main sections, with subsections and numbered steps to facilitate ease of use:

The Proposed Architecture section covers setup, installing the test framework, creating the first test, and defining the server.

The Develop and Test Server section describes preparing the server for content testing, as well as testing and adding support to the server.

The Develop and Test Controller section provides detail on developing the initial controller so that it supports standard REST actions (index, show, update, and destroy), and on refactoring the controller after this development.

 

Proposed Architecture

 

Let’s start with the general API architecture. Some elements of MVC patterns can be successfully reused here. Strictly speaking, I am going to focus primarily on one element of that pattern:  the Controller. The architecture I am going to develop is represented by the gray rectangle in the schema below:

 

image00

 

Set Up Architecture

 

1. Create an application directory:

 

 

2.  Initialize the git repository:

 

 

3. Initialize the npm application:

 

 

4. Accept all proposed default values.

 

Install Test Framework, Create First Test, Define Server

Mocha’s framework will work well for this project.

 

1.  Install Mocha:

 

 

2. Create an empty directory for tests:

 

 

3. Configure Mocha as your default test framework in the package.json:

 

 

The above steps enable you to run tests with the default npm command:

 

 

At this point, you should see something like No test files found in the output. This is expected. We have not created any tests so far. That is the next step.

 

4. Install the supertest library for request testing:

 

 

5. Create the first test:

test/server.js:

 

 

6. Run the first test and see this expected error message:

 

 

7. Configure server instance:

I will use the Restify package for building the web API:

 

 

Here is the basic trivial server just to make our only test green:

 

 

 

You can see that this is a very trivial server definition. It has only one action defined, GET /posts, which always returns an empty object. If all steps have been done properly, then npm test will output:

 

 

Now we have a server that responds to GET /posts properly. It is the perfect time to make our first manual integration test to check that all the pieces we have created fit together properly.

 

8. Create script that runs the server:

./start.js:

 

 

9. Update package.json to define it as an application start script:

package.json:

 

 

10. Run the server itself:

 

 

The server should be running and ready to accept requests on the port 8080:

 

 

 

It will always respond with an empty object as expected:

 

 

Now we have an API server that is ready to respond to our requests.

 

Develop and Test Server

 

According to our proposed architecture, the server should redirect all requests to the controller, PostsController in our case. It will be responsible for handling requests, gathering all necessary data, and rendering results.

 

Prepare Server for Content Testing

 

1. Extract the controller from the current server implementation:

app/controllers/posts.js:

 

 

This is the simplest possible controller version.

 

2. Modify our server to use the controller:

 

 

Now we have a controller instance that has been created INSIDE the server. But to be able to write isolated unit tests of the server, we need some way to pass a fake controller to the server and then ensure that all methods on that fake controller are within proper parameters.

 

3. Modify the server to be able to accept controller instance:

app/server.js:

 

 

As a result of this step, we have defined the server factory rather than the server definition. This server factory created a server instance based on the controller parameters that it accepts.

 

Test Server

 

1. Modify the server test and specify fake controller instance:

test/server.js:

 

 

You can see above that posts is a simple, plain object used as a controller stub object that has only one method, index, defined on it. That makes it possible to control both the results that are returned to the server from the controller, and the params that are passed from the server to the controller. (More detail on this below.) If the steps have been done properly, all tests will be green now.

 

2. Run test:

 

 

3. Modify our start script and pass real PostsController instance to the server instance:

./start.js:

 

 

4. Make a simple integration test to check that nothing is broken:

Run server:

 

 

Send a test request to the server:

 

 

An empty object is returned, which is exactly what was expected.

Now we have everything prepared for content testing, specifically making sure that the server properly serializes data that is returned from the controller and responds with that data.

 

5. Update test so it becomes red:

test/server.js:

 

 

6. Run npm test and see an error:

 

 

It looks like the server does not return post data. The server needs to be modified.

 

7. Update server:

 

 

If you run npm test now, you should see that all tests are green. Nice!

 

Add Support to Create New Post Instance

 

Now it’s time to add support for one more action to our server, POST /posts, that will create a new post instance.

 

1. Create test:

test/server.js:

 

 

2. Run npm test and see the following error:

 

 

This result is actually expected. POST /posts must be defined on the server to fix the test.

 

3. Define POST /posts on the server:

app/server.js:

 

 

If we run npm test at this point, we will still see an error:

 

 

It looks like the params we sent to the server were not parsed properly.

 

4. Plug body parser into the server:

app/server.js:

 

 

5. Run npm test again. Everything should be fine.

 

The next action I am going to add is GET /posts/:id. This action is different from the previous two. It’s tricky in that the API consumer might specify nonexistent post identities that the server must handle gracefully.

For now, let’s implement a simple action version that does not handle situations when posts do not exist.

 

6. Create test as usual:

test/server.js

 

 

7. Run npm test and get an error:

 

 

The resource is not found. We need to define the action on the server.

 

8. Define action on the server:

app/server.js:

 

 

9. Run test. Now everything should be green!

But what about cases when there is no post with the specified ID? The server should obviously return NotFound (404) HTTP status in this case. Let’s continue by addressing this situation.

 

10. Add test:

test/server.js:

 

 

11. Run npm test again and get an error:

 

 

This error occurred because the promise in the controller stub was rejected. We need to modify the server to handle this circumstance.

 

12. Correct error:

app/server.js:

 

 

13: Run tests again. They should all be green.

The next action in line involves update: POST /posts/:id. Similar to the previous action, we must develop a clear path first, assuming that the correct post ID is specified. We will consider situations when an invalid ID is specified later.

 

14. Create test first as usual:

 

 

15. Run test and see an expected error:

 

 

16. Define missing method:

app/server.js:

 

 

Now we have an updated server action that is capable of handling existing resources. It’s time to address cases when the identifier of nonexistent posts is specified.

 

17. Create the test:

test/server.js:

 

 

18. Run npm test and get an error:

 

 

This result is expected. Rejected promises have not been addressed yet.

 

19. Add error handling to the server action:

 

 

20. Run tests again. They are green!

At this point, only one action has not been implemented: DELETE /posts/:id. Now it’s time to fill this gap.

Here is the code for the action test:

 

 

21. Run action test. Get an error.

 

22. Define action on the server:

 

 

23. Run tests again. Green!

Now let’s handle cases when there is no post with the specified ID.

Here is the test code:

test/server.js:

 

 

24. Run the test. Get timeout error.

 

25. Update server:

app/server.js:

 

 

Everything is green.

Now we have a fully workable server. It properly redirects request data to the controller and writes serialized results to the response. It does not yet access ES instances and returns dummy data. This function is addressed below.

 

Develop and Test Controller

 

This section concerns specifically the PostsController. It will work with an ES client. We can assume that the ES client is well-tested so we do not have to test it ourselves. Only the methods of the controller in isolation should be tested. To do so, we need to pass the client stub to the controller instance to be able to verify that the correct methods were called on the stub, and the returned data was properly handled.

 

Prepare Controller for Testing

 

1. Update PostsController definition so it accepts client instance from outside:

app/controllers/posts.js:

 

 

2. Install another test library:

 

 

3. Introduce the ES client stub into the posts controller test:

test/controllers/posts.js:

 

 

4. Run tests. Everything should be green.

Now we have prepared the basis for PostsController development.

 

Test Controller

 

1. Write our first test – index action.

Here is the test code:

 

 

We see that the possible ES response is imitated here.

 

2. Run the first test and see a failure. Our current PostsController always returns an empty object.

 

3. Correct failure:

 

 

4. Run test again. Green!

 

Test Parameters

 

We just tested that PostsController parsed the ES result properly, but we also need to test that it passes correct params to the client. Two params of the ES client method need to be specified: search: index and type.

 

1. Add params to the PostsController constructor:

app/controllers/posts.js:

 

 

2. Specify test params in the PostsController specs:

test/controller/posts.js:

 

 

3. Run test again. Green! Nothing is broken.

 

4. Verify that we specified those params properly in test when client.search method is called.

I am going to use sinon for spying on method calls:

 

 

I am using should-sinon for should – like asserts:

 

 

Specify these in the controller test:

test/controllers/posts.js:

 

 

Now we can write a params verification test:

test/controllers/posts.js:

 

 

5. Run npm test. See the following failure:

 

 

6. Update the controller:

app/controllers/posts.js:

 

 

The tests should be green.

 

Create and Run Manual Integration Tests

 

Now is a good time to run simple manual integration tests to be sure that all the pieces we have created correspond properly. Each piece is covered by its own unit test. We cannot be sure they all interact correctly without integration testing. It would be best to create manual integration tests for this purpose, but that is beyond the scope of this article.

To complete the following steps, you should have ES service installed locally and be running on default 9200 port.

 

1. Create index ( ‘node_api’ ):

 

 

Expected output:

 

 

2. Create post example:

 

 

Expected output:

 

 

3. Install elasticsearch npm package:

 

 

4. Create ES client instance:

./app/client.js:

 

 

5. Update start script with the real index and type names:

./start.js:

 

 

6. Run server:

 

 

7. Make test request:

 

 

If all steps have been done properly, you should see something like this in the output:

 

 

Our integration test was successful. We can continue adding methods to the controller, knowing that the server has been configured properly and calls proper controller methods.

 

Implement Post Indexing in ES

 

1. Create test code:

 

 

Two tests are defined above. In real situations, these tests would consist of two interactions. I joined them into one interaction here for simplicity’s sake.

 

2. Run tests and see errors

 

3. Add indexing support to the controller:

 

 

The tests will be green if steps have been done properly.

 

Implement “Show” Action

 

The next controller action is show. Similar to GET /show/:id, this controller action should handle situations when a post with a specified identifier does not exist. We will take care of that later. Now, let’s start from the simplified action version, assuming that only a correct identifier can be specified.

 

1. Create test first as usual:

test/controllers/posts.js:

 

 

2. Run npm test, get error.

If you run npm test now, you will see an error because method show has not been defined on the PostsController.

 

3. Correct error:

app/controllers/posts.js:

 

 

4. Run npm test again. All tests should be green.

The PostsController is now able to find the post and return its content.

But if the identifier of a nonexistent post was specified, the controller will fail. We need to handle this exceptional situation properly.

 

5. Create test for nonexistent post identifier:

test/controllers/posts.js:

 

 

6. Run test and get an error.

 

7. Update controller:

app/controllers/posts.js:

 

 

Now we have fully implemented show action.

 

Enable “Update” Function

 

The next action concerns update. As in the case of show action, we need to address situations when a nonexistent post identifier is passed to the action. Also similar to the show, we will handle that later and start from a simplified version.

 

1. Create the test code:

 

 

2. Run npm test and get these errors:

 

 

So, update is not yet a function. Let’s fix this.

 

3. Define update method:

app/controllers/posts.js:

 

 

The errors should be corrected.

Now it’s time to address situations when an identifier of a nonexistent resource is specified.

 

4. Create test for nonexistent ID specification:

 

 

5. Run test and see a failure.

 

6. Update the definition of the method update:

 

 

7. Run tests. They should be green.

 

Enable “Destroy” Function

 

This is the final REST action to be implemented. As with previous actions, we need to handle situations when the ID of the non-existing resource is specified as a parameter of the action. Also following our previous methods, we will implement a simple action version first and handle non-existing sources second.

 

1. Create the test code:

test/controllers/posts.js:

 

 

2. Run npm test. See these errors:

 

 

3. Define destroy action:

 

 

4. Run test; It should be green.

We are now able to destroy the post with the specified identifier.

 

5. Create test code for nonexistent resource:

 

 

6. Check functionality:

 

 

7. Run tests. They should be green.

Now we have completed all API functionality. Posts can be created, deleted, updated, and listed. The next section covers clean up.

 

Refactor Controller

 

Refactoring is a safe and easy operation in our case because the tests cover it.

Right now, we have a repetitive pattern in our app/controllers/posts.js:

 

 

Let’s try to DRY it and clean it up.

 

1. Extract all such patterns into a dedicated class:

app/lib/resource.js:

 

 

Our controller has now been simplified a bit:

 

 

We have extracted all our interactions into a special, dedicated Resource class. But results parsing is still in the controller. Let’s address this.

 

2. Extract results parsing into a special Parser class:

app/lib/parser.js:

 

 

Here is the result:

 

 

Our controller looks much better now!

 

Summary

 

We have now completed a Node.js API for an ES by following step-by-step instructions led by testing. The result is a relatively simple API with reliable test coverage. Each of the steps described is trivial and might be done quite easily without any debugging efforts. The TDD approach I have detailed results in better, cleaner code when developing Node.js applications. Once you try it, you will see it is also faster than creating code before the test.

Consulting & Advisory Software Development Software Testing Test Automation

Latest Insights in Consulting & Advisory

The Rise of Kotlin – Moving Away from Java for Android Development

Kotlin is a programming language for the Java Virtual Machine that’s able to be used in any scenarios that currently…

Introducing our Sphere Heroes Program – Artem Korenev – Employee of the month

At Sphere, employee recognition is a key component of our corporate culture. We believe in celebrating the successes of our…

Write For Sphere

Are you a writer with tech expertise? Then we want to hear from you! Here are a few guidelines for…

View All Articles arrow

We are here to help:

checkmarkto become a customer checkmarkto become an investor checkmarkto send a media inquiry checkmarkto join our team checkmarkto simply say ‘hi’
Get in Touch