Building an API Gateway

This month I started work on my own API Gateway. I have started work on this, so that I can have a single point of contact, for many of my other scripts and services I have built and will build in the future.

So far, it is a very simplistic system, as I am still learning how everything works. At this time, it can only do a few things, such as, parse the incoming path, respond to pings, check a user is authenticated and upload a file to hard-coded location. Though I do plan to have this perform many other actions. I am going to make this a contact point for Wolf Bot, so that I have a single point for logging authentication actions on my servers, and if I can work out how to do it, build my own log forwarder and have a single log processor, without using external tools.

As part of trying to learn as much as I can when building this, I have not used any external modules/libraries (so far), for example I am not using express, I am instead using the built-in http module of Node and handling all the request processing myself, which I have found it more difficult than I expected. I spent a good amount of time earlier this week trying to work out how to correctly handle incoming data in the body. For example, to upload a file, I have to create an array of buffers and then concatenate this, so that I can write to disk, but I also had to create a system for checking the size of the chunks being processed, so that I can implement and enforce a maximum file size, without knowing the size of the file arriving prior.

The authentication for the API is key based, I am making sure to store these correctly, by both salting and hashing the key before storing the key to disk, so that there is no plain text copy of the authentication passwords. Though, I am currently storing these in a JSON array, but this will be moved to SQLite or some other database once I get to that implementation step. Which hopefully will be next week; though I want to not use an external library for connecting to databases, so will need to see if that can be done.

As part of building this, I had to also build some kind of logging mechanism, for both audit and troubleshooting. I thus have built a simple module that takes ongoing events and logs them to disk independently of the current actions. As part of creating this API, I am also (mostly) following standard best practises, such as https://www.gov.uk/guidance/gds-api-technical-and-data-standards. Therefore, I am using https response codes and requests follow the POST, GET, PUT etc, formatting correctly, finally the format for my API (To be finalised) is (https://host:port/api/v1/ping) and using headers for authentication, so when connecting over HTTPS, these are relatively secure.