|
|
|
@ -57,7 +57,7 @@ support for hardware and software is limited to an x86 Linux based
|
|
|
|
|
machine running Nginx. I decided to run a virtual machine on Amazon |
|
|
|
|
Web Services. |
|
|
|
|
|
|
|
|
|
- [Amazon Web Services](https://aws.amazon.com/) |
|
|
|
|
- [Amazon Web Services](https://aws.amazon.com/) (A.W.S.) |
|
|
|
|
|
|
|
|
|
## Software Requirements |
|
|
|
|
|
|
|
|
@ -71,6 +71,224 @@ Services. Links for various parts of the set-up are below:
|
|
|
|
|
- [https://ubuntu.com/](https://ubuntu.com/) |
|
|
|
|
- [Nginx](https://www.nginx.com/) |
|
|
|
|
- [Python Flask](https://flask.palletsprojects.com/en/2.0.x/) |
|
|
|
|
- [Gunicorn](https://gunicorn.org/) |
|
|
|
|
- [Supervisor](https://pypi.org/project/supervisor/) (Python) |
|
|
|
|
- [SQLite Database](https://sqlite.org/index.html) |
|
|
|
|
- [SQL Alchemy](https://www.sqlalchemy.org/) (O.R.M.) |
|
|
|
|
- [Swagger REST API](https://swagger.io/) |
|
|
|
|
|
|
|
|
|
## Software Set-up |
|
|
|
|
|
|
|
|
|
Below is an overview of the files and directory structure. |
|
|
|
|
|
|
|
|
|
```shell |
|
|
|
|
app/ |
|
|
|
|
├── api.py <-------- REST API entry point. |
|
|
|
|
├── app.py <------- 'Main' program entry point. |
|
|
|
|
├── build_database.py <-- Builds database (use first). |
|
|
|
|
├── config.py <----- Program config's for Flask and Swagger. |
|
|
|
|
├── models <-------- Objects which database rows are mapped to. |
|
|
|
|
├── __pycache__ |
|
|
|
|
├── readings.db <--- The SQLite database. |
|
|
|
|
├── services <------ 'Business logic' for api.py entry points. |
|
|
|
|
├── static <------- Images/Favicons/Style Sheets. |
|
|
|
|
├── swagger.yml <--- YAML config. file for Swagger (REST API). |
|
|
|
|
└── templates <----- HTML views. |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
### The Program (Python and Virtual Environment) |
|
|
|
|
|
|
|
|
|
**I have omitted instructions on creating user accounts beyond the |
|
|
|
|
defaults** because that comes with a fair bit of context specific |
|
|
|
|
information which is beyond the scope of this documentation. On top of |
|
|
|
|
that, the project's budget did not allow for extended research on |
|
|
|
|
various Cloud Platforms and server architecture, in-general. |
|
|
|
|
|
|
|
|
|
After you have established a V.M. on A.W.S. or your own server, you |
|
|
|
|
will need to clone the repository (**I am assuming this is via SSH**). |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
# Adjust the directories to how you prefer it. These are just my |
|
|
|
|
defaults. |
|
|
|
|
mkdir www |
|
|
|
|
cd www |
|
|
|
|
git clone http://git.abbether.net/return-to-ritherdon/midpoint.git |
|
|
|
|
cd midpoint |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
From there, you will need to create a Python virtual environment, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
# Adjust the path to suit to environment. |
|
|
|
|
python3 -m venv ~/repos/midpoint/venv |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
As an aside, I created a virtual environment with `python3 -m venv |
|
|
|
|
~/www/midpoint/my-env` whilst in development. This environment might |
|
|
|
|
still be around the production server, used during the |
|
|
|
|
exhibition. **If you are completing a fresh install, this is not |
|
|
|
|
relevant to you. Please ignore.** |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### The Server |
|
|
|
|
|
|
|
|
|
The host name for the server used during the exhibition was |
|
|
|
|
`piapi`. So, I will use that in the documentation below. **Please make |
|
|
|
|
sure you replace `piapi` with the host name you are using.** |
|
|
|
|
|
|
|
|
|
I used Nginx and Gunicorn for the exhibition so that is all this |
|
|
|
|
documentation will focus on. Midpoint should work on other servers |
|
|
|
|
(like Apache) but I have not tested it. |
|
|
|
|
|
|
|
|
|
The Nginx configuration file is located at: |
|
|
|
|
|
|
|
|
|
- `/etc/nginx/sites-enabled/piapi` |
|
|
|
|
|
|
|
|
|
You can access the file with `sudo nano |
|
|
|
|
/etc/nginx/sites-enabled/piapi` when logged into the server. |
|
|
|
|
|
|
|
|
|
Because Nginx is responsible for the static side of the website, |
|
|
|
|
Gunicorn is needed to operate the Python side of it. Nginx knows |
|
|
|
|
how/when to pass control over to Gunicorn because it is configured in |
|
|
|
|
the … sites-enables/piapi configuration file (mentioned above). With |
|
|
|
|
that said, Gunicorn still needs to be set-up for everything to run |
|
|
|
|
properly. It is worth pointing out, the "static" part of the site will |
|
|
|
|
still run when Gunicorn is not running but that is a practically |
|
|
|
|
useless feature. To run Gunicorn, enter the following command into the |
|
|
|
|
terminal (of logged in V.M), |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
# You can adjust the '-w 3' if you have a more powerful server. Also, |
|
|
|
|
the 'connext_app' related to the use of the Swagger API code. |
|
|
|
|
gunicorn -w 3 server:connex_app |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
The -w 3 bit refers to the total number of workers (processors) |
|
|
|
|
Gunicorn will use whilst running. The general rule to work that out it |
|
|
|
|
/(number of processors * 2) + 1. At the time of writing, the V.M. is a |
|
|
|
|
very basic one-core machine so that is why three workers is used. |
|
|
|
|
|
|
|
|
|
One problem with Gunicorn is it blocks the terminal when using it in |
|
|
|
|
its default way. To get around this, I have install Supervisor to |
|
|
|
|
manage it. Supervisor handles the auto-restart after a crash and |
|
|
|
|
background-task management of it too. To install it, run the following |
|
|
|
|
command, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
sudo apt install supervisor |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
When Supervisor is installed, you need to create a config. file. The |
|
|
|
|
file for this project is stored at |
|
|
|
|
`/etc/supervisor/conf.d/spiapi.conf`. The log files (specified in the |
|
|
|
|
.conf) file are, |
|
|
|
|
|
|
|
|
|
- `/var/log/midpoint/midpoint.err.log` |
|
|
|
|
- `/var/log/midpoint/midpoint.out.log` |
|
|
|
|
|
|
|
|
|
You will to recreate the directory they are stored in if you are |
|
|
|
|
installing this site to a new system. The easiest way to do that is to |
|
|
|
|
run the following command, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
sudo mkdir -p /var/log/roaming-light-api |
|
|
|
|
sudo touch /var/log/midpoint/midpoint.err.log |
|
|
|
|
sudo touch /var/log/midpoint/midpoint.out.log |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
When you have finished making these changes, you will need to restart |
|
|
|
|
Supervisor. To do that, run the following command, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
sudo supervisor reload |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
It might take a little while for the service to restart so if you |
|
|
|
|
still see "Bad Gateway" messages in the browser, that might be why. |
|
|
|
|
|
|
|
|
|
You might find Gunicorn is not installed in the virtual-environment, |
|
|
|
|
it that is the case, you should find it at, |
|
|
|
|
|
|
|
|
|
- `home/ubuntu.local/bin/gunicorn` |
|
|
|
|
|
|
|
|
|
This might, also, mean you need to use apt and not pip3. |
|
|
|
|
|
|
|
|
|
The config. file for Supervisor should look something like the |
|
|
|
|
following, (remember to adjust the places where `ubuntu` is with your |
|
|
|
|
servers account name -- if it differs from `ubuntu`) |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
[program:midpoint] |
|
|
|
|
directory=/home/ubuntu/www/midpoint/app |
|
|
|
|
command=/home/ubuntu/.local/bin/gunicorn -w 3 server:connex_app |
|
|
|
|
user=ubuntu |
|
|
|
|
autostart=true |
|
|
|
|
autorestart=true |
|
|
|
|
stopasgroup=true |
|
|
|
|
killasgroup=true |
|
|
|
|
stderr_logfile=/var/log/midpoint/midpoint.err.log |
|
|
|
|
stdout_logfile=/var/log/midpoint/midpoint.out.log |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
The config. file for Nginx should look like the following, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
server { |
|
|
|
|
listen 80; |
|
|
|
|
server_name 35.176.235.94; |
|
|
|
|
|
|
|
|
|
location /static { |
|
|
|
|
alias /home/ubuntu/www/midpoint/app/static; |
|
|
|
|
} |
|
|
|
|
|
|
|
|
|
location / { proxy_pass http://localhost:8000; include |
|
|
|
|
/etc/nginx/proxy_params; proxy_redirect off; } } |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
**I did not set-up the server to use HTTPS because it was outside the |
|
|
|
|
scope of the project.** Again, replace `ubuntu` with your servers |
|
|
|
|
username. |
|
|
|
|
|
|
|
|
|
### Flask & Swagger |
|
|
|
|
|
|
|
|
|
To access the A.P.I. user-interface (via Swagger), enter |
|
|
|
|
`http://0.0.0.0:5000/api/ui/`. |
|
|
|
|
|
|
|
|
|
- [Real Python](https://realpython.com/flask-connexion-rest-api/#using-connexion-to-add-a-rest-api-endpoint) |
|
|
|
|
|
|
|
|
|
This site provides a good walk-through for setting-up a website using |
|
|
|
|
the Flask framework. Real Python, also, provides guides on creating |
|
|
|
|
websites with Python and its various website frameworks. |
|
|
|
|
|
|
|
|
|
I have imported the `connexion` project into this one. By doing this, |
|
|
|
|
I can use Swagger and its U.I. When you have the server running, go to |
|
|
|
|
`localhost:5000/api/swagger`. When there, you can test the A.P.I. out |
|
|
|
|
by sending it "gets" and "posts". |
|
|
|
|
|
|
|
|
|
![Swagger Screenshot](attachments/swagger-ui.png) |
|
|
|
|
|
|
|
|
|
The Swagger endpoint are configured in a YAML file at, |
|
|
|
|
|
|
|
|
|
- [midpoint/app/swagger.yml](https://git.abbether.net/return-to-ritherdon/midpoint/src/branch/unstable/app/swagger.yml) |
|
|
|
|
|
|
|
|
|
### Database and O.R.M. |
|
|
|
|
|
|
|
|
|
The project uses a SQLite database to store light reading and any |
|
|
|
|
device status updates. To write to the database, I used SQL Alchemy as |
|
|
|
|
a Object Relational Mapper (O.R.M). The database is called, |
|
|
|
|
|
|
|
|
|
- `readings.db` |
|
|
|
|
|
|
|
|
|
Before you can start the program, you will need to build the |
|
|
|
|
database. To do that, you will need to run the following commands, |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
# Make sure you are in the 'app' directory. Adjust the 'cd' path to |
|
|
|
|
# match your set-up. |
|
|
|
|
cd ~/www/midpoint/app/ |
|
|
|
|
|
|
|
|
|
python3 build-database.py |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
This will clear the database if one is already set-up and then build a |
|
|
|
|
new one. |
|
|
|
|