diff --git a/midpoint/attachments/swagger-ui.png b/midpoint/attachments/swagger-ui.png new file mode 100644 index 0000000..4b679b2 Binary files /dev/null and b/midpoint/attachments/swagger-ui.png differ diff --git a/midpoint/rtr-midpoint.md b/midpoint/rtr-midpoint.md index 355d31f..aaafc82 100644 --- a/midpoint/rtr-midpoint.md +++ b/midpoint/rtr-midpoint.md @@ -57,7 +57,7 @@ support for hardware and software is limited to an x86 Linux based machine running Nginx. I decided to run a virtual machine on Amazon Web Services. -- [Amazon Web Services](https://aws.amazon.com/) +- [Amazon Web Services](https://aws.amazon.com/) (A.W.S.) ## Software Requirements @@ -71,6 +71,224 @@ Services. Links for various parts of the set-up are below: - [https://ubuntu.com/](https://ubuntu.com/) - [Nginx](https://www.nginx.com/) - [Python Flask](https://flask.palletsprojects.com/en/2.0.x/) +- [Gunicorn](https://gunicorn.org/) +- [Supervisor](https://pypi.org/project/supervisor/) (Python) - [SQLite Database](https://sqlite.org/index.html) +- [SQL Alchemy](https://www.sqlalchemy.org/) (O.R.M.) - [Swagger REST API](https://swagger.io/) +## Software Set-up + +Below is an overview of the files and directory structure. + +```shell +app/ +├── api.py <-------- REST API entry point. +├── app.py <------- 'Main' program entry point. +├── build_database.py <-- Builds database (use first). +├── config.py <----- Program config's for Flask and Swagger. +├── models <-------- Objects which database rows are mapped to. +├── __pycache__ +├── readings.db <--- The SQLite database. +├── services <------ 'Business logic' for api.py entry points. +├── static <------- Images/Favicons/Style Sheets. +├── swagger.yml <--- YAML config. file for Swagger (REST API). +└── templates <----- HTML views. +``` + +### The Program (Python and Virtual Environment) + +**I have omitted instructions on creating user accounts beyond the +defaults** because that comes with a fair bit of context specific +information which is beyond the scope of this documentation. On top of +that, the project's budget did not allow for extended research on +various Cloud Platforms and server architecture, in-general. + +After you have established a V.M. on A.W.S. or your own server, you +will need to clone the repository (**I am assuming this is via SSH**). + +```bash +# Adjust the directories to how you prefer it. These are just my +defaults. +mkdir www +cd www +git clone http://git.abbether.net/return-to-ritherdon/midpoint.git +cd midpoint +``` + +From there, you will need to create a Python virtual environment, + +```bash +# Adjust the path to suit to environment. +python3 -m venv ~/repos/midpoint/venv +``` + +As an aside, I created a virtual environment with `python3 -m venv +~/www/midpoint/my-env` whilst in development. This environment might +still be around the production server, used during the +exhibition. **If you are completing a fresh install, this is not +relevant to you. Please ignore.** + + +### The Server + +The host name for the server used during the exhibition was +`piapi`. So, I will use that in the documentation below. **Please make +sure you replace `piapi` with the host name you are using.** + +I used Nginx and Gunicorn for the exhibition so that is all this +documentation will focus on. Midpoint should work on other servers +(like Apache) but I have not tested it. + +The Nginx configuration file is located at: + +- `/etc/nginx/sites-enabled/piapi` + +You can access the file with `sudo nano +/etc/nginx/sites-enabled/piapi` when logged into the server. + +Because Nginx is responsible for the static side of the website, +Gunicorn is needed to operate the Python side of it. Nginx knows +how/when to pass control over to Gunicorn because it is configured in +the … sites-enables/piapi configuration file (mentioned above). With +that said, Gunicorn still needs to be set-up for everything to run +properly. It is worth pointing out, the "static" part of the site will +still run when Gunicorn is not running but that is a practically +useless feature. To run Gunicorn, enter the following command into the +terminal (of logged in V.M), + +```bash +# You can adjust the '-w 3' if you have a more powerful server. Also, +the 'connext_app' related to the use of the Swagger API code. +gunicorn -w 3 server:connex_app +``` + +The -w 3 bit refers to the total number of workers (processors) +Gunicorn will use whilst running. The general rule to work that out it +/(number of processors * 2) + 1. At the time of writing, the V.M. is a +very basic one-core machine so that is why three workers is used. + +One problem with Gunicorn is it blocks the terminal when using it in +its default way. To get around this, I have install Supervisor to +manage it. Supervisor handles the auto-restart after a crash and +background-task management of it too. To install it, run the following +command, + +```bash +sudo apt install supervisor +``` + +When Supervisor is installed, you need to create a config. file. The +file for this project is stored at +`/etc/supervisor/conf.d/spiapi.conf`. The log files (specified in the +.conf) file are, + +- `/var/log/midpoint/midpoint.err.log` +- `/var/log/midpoint/midpoint.out.log` + +You will to recreate the directory they are stored in if you are +installing this site to a new system. The easiest way to do that is to +run the following command, + +```bash +sudo mkdir -p /var/log/roaming-light-api +sudo touch /var/log/midpoint/midpoint.err.log +sudo touch /var/log/midpoint/midpoint.out.log +``` + +When you have finished making these changes, you will need to restart +Supervisor. To do that, run the following command, + +```bash +sudo supervisor reload +``` + +It might take a little while for the service to restart so if you +still see "Bad Gateway" messages in the browser, that might be why. + +You might find Gunicorn is not installed in the virtual-environment, +it that is the case, you should find it at, + +- `home/ubuntu.local/bin/gunicorn` + +This might, also, mean you need to use apt and not pip3. + +The config. file for Supervisor should look something like the +following, (remember to adjust the places where `ubuntu` is with your +servers account name -- if it differs from `ubuntu`) + +```bash +[program:midpoint] +directory=/home/ubuntu/www/midpoint/app +command=/home/ubuntu/.local/bin/gunicorn -w 3 server:connex_app +user=ubuntu +autostart=true +autorestart=true +stopasgroup=true +killasgroup=true +stderr_logfile=/var/log/midpoint/midpoint.err.log +stdout_logfile=/var/log/midpoint/midpoint.out.log +``` + +The config. file for Nginx should look like the following, + +```bash +server { + listen 80; + server_name 35.176.235.94; + + location /static { + alias /home/ubuntu/www/midpoint/app/static; + } + + location / { proxy_pass http://localhost:8000; include + /etc/nginx/proxy_params; proxy_redirect off; } } +``` + +**I did not set-up the server to use HTTPS because it was outside the +scope of the project.** Again, replace `ubuntu` with your servers +username. + +### Flask & Swagger + +To access the A.P.I. user-interface (via Swagger), enter +`http://0.0.0.0:5000/api/ui/`. + +- [Real Python](https://realpython.com/flask-connexion-rest-api/#using-connexion-to-add-a-rest-api-endpoint) + +This site provides a good walk-through for setting-up a website using +the Flask framework. Real Python, also, provides guides on creating +websites with Python and its various website frameworks. + +I have imported the `connexion` project into this one. By doing this, +I can use Swagger and its U.I. When you have the server running, go to +`localhost:5000/api/swagger`. When there, you can test the A.P.I. out +by sending it "gets" and "posts". + +![Swagger Screenshot](attachments/swagger-ui.png) + +The Swagger endpoint are configured in a YAML file at, + +- [midpoint/app/swagger.yml](https://git.abbether.net/return-to-ritherdon/midpoint/src/branch/unstable/app/swagger.yml) + +### Database and O.R.M. + +The project uses a SQLite database to store light reading and any +device status updates. To write to the database, I used SQL Alchemy as +a Object Relational Mapper (O.R.M). The database is called, + +- `readings.db` + +Before you can start the program, you will need to build the +database. To do that, you will need to run the following commands, + +```bash +# Make sure you are in the 'app' directory. Adjust the 'cd' path to +# match your set-up. +cd ~/www/midpoint/app/ + +python3 build-database.py +``` + +This will clear the database if one is already set-up and then build a +new one.