I decided to self delegate myself the role of setting up and developing the backend for project Six, this involved me learning Node.js, web sockets, how to implement a server-side MQTT broker and know enough about three.js to plug things together on the client side.

Node js

Node.js is a serve side implementation of Javascript, I set out to learn Node.js for Six and another project lucid with little prior experience of Javascript, previously only using basic functionality and some Jquery snippets. Node.js gave us the ability to develop a system that could receive information from our Arduino setup, deal with the logic and calculation and then send relevant data for the visualisation through sockets to the web client. Five weeks ago I'd not had a single experience with Node.js but by the end of our project, Six had a fully functioning database enabled backend running on Node.js. All of the code discussed here is available on our Github repo here, but remember this is my first node project so be nice. I struggled a little to modularise my code effectively leaving anything that required database calls or socket functionality within my main app file, however i did manage to utilise the ability and had 4 modules with functions exported for use with the main app file. The code is a little spaghetti but overall still only about 800 lines and thats with comments and logs included.

I decided to use the express framework for Node.js which would allow me to scale the project nicely if/when we needed more than a single page. Acting a a simple http server express deals with requests sending all users that access the web app on port :8080 to '/' our index page, when we needed I added another route for '/multi' for our example view of a system with more users.

MQTT Broker

The MQTT broker was to be one of the most important links in our chain providing the support for the Arduino controlled devices to communicate with the server in real-time using a pub/sub pattern. For our previous project ESN we used a local version of the RSMB provided by IBM, that worked great in that project but required all devices to be connected to a local network as such meant it was only available within one building. Six was to be a much larger endeavour, be real world applicable and allow our Arduino devices to be mobile and still connected to the system, our Node.js setup worked towards this end.

The Node.js MQTT implementation worked like any other broker allowing MQTT clients to publish on any channels, it will pass these packets to all clients who are subscribed on that same channel. All though not technically subscribed to the packets the MQTT Sever is passing packets published onto other client and these can be read, manipulated if needed and stored stored while still passing them to all clients that are subscribed. We had our Arduino devices publishing on [id]/gpsLong and [id]/gpsLat for there current positions this information was then parsed by the server and stored in a temporary for use and then overwritten each time there is a new position.

Each time a new relevant packet is published it's check against every know position of other Arduino's in the system. If any of the users in the system are within our threshold of another user (we used 0.0003 Lat & Long which is Equal to 20meters apart) it will register as a connection. The Arduino's involved are ran through a function to check their unique connection id. The connection between all users that are close is increased as is the Max connection for each User, these values are then databased using MongoDB. MongoDB is a non blocking Javascript database that is fully functional within a server-side environment. There was a slight Learning curve for learning MongoDB and I'm still not sure i'm doing it the right was but it worked for what we needed.

As the Node server is running continuously on my VPS server any Arduino that has access to the internet can publish/subscribe MQTT topics to the specific IP on port 8085, this allows for the access on the move that we needed when combined with 3G.

MQTT Publish Client

I was able to integrate a simple MQTT client that sits within the Node.js app. Now the client was available server-side I could simply use a function to connect the client to our MQTT broker and another function throughout out server that would pass in a topic and a packet of information to be sent whenever necessary. We used this publish client a number of times both for automated response when our users make connections and when manual buttons are pressed within the user interface online. When buttons are pressed on the user interface a corresponding socket connection is made and then the server responses by running the publish client functions with relevant parameters.

mosquitto.js

After looking into MQTT in preparation for this build I and other on the course came across an open source Javascript MQTT client named mosquito.js. It seemed we had stuck Gold as a Javascript MQTT client could possibly cut out the need for a web socket solution and possibly even a our Node.js system. Although this library is listed on the MQTT site it unfortunately only runs under very strict circumstances requiring the mosquito broker, a lighttpd sever and a specific web sockets plugin so became unnecessary, all this discovered after hours of trying to hack it together. I was unable to find any other javascript based clients for MQTT so this is why i used the socket solution discussed with the frontend post, however I would really appreciate if you know of something, give me a shout over on twitter @Sam_Billingham