Fork 0
Browse Source

populate midpoint doc.

Craig Oates 2 years ago
  1. BIN
  2. 220


Binary file not shown.


Width:  |  Height:  |  Size: 56 KiB


@ -57,7 +57,7 @@ support for hardware and software is limited to an x86 Linux based
machine running Nginx. I decided to run a virtual machine on Amazon
Web Services.
- [Amazon Web Services](https://aws.amazon.com/)
- [Amazon Web Services](https://aws.amazon.com/) (A.W.S.)
## Software Requirements
@ -71,6 +71,224 @@ Services. Links for various parts of the set-up are below:
- [https://ubuntu.com/](https://ubuntu.com/)
- [Nginx](https://www.nginx.com/)
- [Python Flask](https://flask.palletsprojects.com/en/2.0.x/)
- [Gunicorn](https://gunicorn.org/)
- [Supervisor](https://pypi.org/project/supervisor/) (Python)
- [SQLite Database](https://sqlite.org/index.html)
- [SQL Alchemy](https://www.sqlalchemy.org/) (O.R.M.)
- [Swagger REST API](https://swagger.io/)
## Software Set-up
Below is an overview of the files and directory structure.
├── api.py <-------- REST API entry point.
├── app.py <------- 'Main' program entry point.
├── build_database.py <-- Builds database (use first).
├── config.py <----- Program config's for Flask and Swagger.
├── models <-------- Objects which database rows are mapped to.
├── __pycache__
├── readings.db <--- The SQLite database.
├── services <------ 'Business logic' for api.py entry points.
├── static <------- Images/Favicons/Style Sheets.
├── swagger.yml <--- YAML config. file for Swagger (REST API).
└── templates <----- HTML views.
### The Program (Python and Virtual Environment)
**I have omitted instructions on creating user accounts beyond the
defaults** because that comes with a fair bit of context specific
information which is beyond the scope of this documentation. On top of
that, the project's budget did not allow for extended research on
various Cloud Platforms and server architecture, in-general.
After you have established a V.M. on A.W.S. or your own server, you
will need to clone the repository (**I am assuming this is via SSH**).
# Adjust the directories to how you prefer it. These are just my
mkdir www
cd www
git clone http://git.abbether.net/return-to-ritherdon/midpoint.git
cd midpoint
From there, you will need to create a Python virtual environment,
# Adjust the path to suit to environment.
python3 -m venv ~/repos/midpoint/venv
As an aside, I created a virtual environment with `python3 -m venv
~/www/midpoint/my-env` whilst in development. This environment might
still be around the production server, used during the
exhibition. **If you are completing a fresh install, this is not
relevant to you. Please ignore.**
### The Server
The host name for the server used during the exhibition was
`piapi`. So, I will use that in the documentation below. **Please make
sure you replace `piapi` with the host name you are using.**
I used Nginx and Gunicorn for the exhibition so that is all this
documentation will focus on. Midpoint should work on other servers
(like Apache) but I have not tested it.
The Nginx configuration file is located at:
- `/etc/nginx/sites-enabled/piapi`
You can access the file with `sudo nano
/etc/nginx/sites-enabled/piapi` when logged into the server.
Because Nginx is responsible for the static side of the website,
Gunicorn is needed to operate the Python side of it. Nginx knows
how/when to pass control over to Gunicorn because it is configured in
the … sites-enables/piapi configuration file (mentioned above). With
that said, Gunicorn still needs to be set-up for everything to run
properly. It is worth pointing out, the "static" part of the site will
still run when Gunicorn is not running but that is a practically
useless feature. To run Gunicorn, enter the following command into the
terminal (of logged in V.M),
# You can adjust the '-w 3' if you have a more powerful server. Also,
the 'connext_app' related to the use of the Swagger API code.
gunicorn -w 3 server:connex_app
The -w 3 bit refers to the total number of workers (processors)
Gunicorn will use whilst running. The general rule to work that out it
/(number of processors * 2) + 1. At the time of writing, the V.M. is a
very basic one-core machine so that is why three workers is used.
One problem with Gunicorn is it blocks the terminal when using it in
its default way. To get around this, I have install Supervisor to
manage it. Supervisor handles the auto-restart after a crash and
background-task management of it too. To install it, run the following
sudo apt install supervisor
When Supervisor is installed, you need to create a config. file. The
file for this project is stored at
`/etc/supervisor/conf.d/spiapi.conf`. The log files (specified in the
.conf) file are,
- `/var/log/midpoint/midpoint.err.log`
- `/var/log/midpoint/midpoint.out.log`
You will to recreate the directory they are stored in if you are
installing this site to a new system. The easiest way to do that is to
run the following command,
sudo mkdir -p /var/log/roaming-light-api
sudo touch /var/log/midpoint/midpoint.err.log
sudo touch /var/log/midpoint/midpoint.out.log
When you have finished making these changes, you will need to restart
Supervisor. To do that, run the following command,
sudo supervisor reload
It might take a little while for the service to restart so if you
still see "Bad Gateway" messages in the browser, that might be why.
You might find Gunicorn is not installed in the virtual-environment,
it that is the case, you should find it at,
- `home/ubuntu.local/bin/gunicorn`
This might, also, mean you need to use apt and not pip3.
The config. file for Supervisor should look something like the
following, (remember to adjust the places where `ubuntu` is with your
servers account name -- if it differs from `ubuntu`)
command=/home/ubuntu/.local/bin/gunicorn -w 3 server:connex_app
The config. file for Nginx should look like the following,
server {
listen 80;
location /static {
alias /home/ubuntu/www/midpoint/app/static;
location / { proxy_pass http://localhost:8000; include
/etc/nginx/proxy_params; proxy_redirect off; } }
**I did not set-up the server to use HTTPS because it was outside the
scope of the project.** Again, replace `ubuntu` with your servers
### Flask & Swagger
To access the A.P.I. user-interface (via Swagger), enter
- [Real Python](https://realpython.com/flask-connexion-rest-api/#using-connexion-to-add-a-rest-api-endpoint)
This site provides a good walk-through for setting-up a website using
the Flask framework. Real Python, also, provides guides on creating
websites with Python and its various website frameworks.
I have imported the `connexion` project into this one. By doing this,
I can use Swagger and its U.I. When you have the server running, go to
`localhost:5000/api/swagger`. When there, you can test the A.P.I. out
by sending it "gets" and "posts".
![Swagger Screenshot](attachments/swagger-ui.png)
The Swagger endpoint are configured in a YAML file at,
- [midpoint/app/swagger.yml](https://git.abbether.net/return-to-ritherdon/midpoint/src/branch/unstable/app/swagger.yml)
### Database and O.R.M.
The project uses a SQLite database to store light reading and any
device status updates. To write to the database, I used SQL Alchemy as
a Object Relational Mapper (O.R.M). The database is called,
- `readings.db`
Before you can start the program, you will need to build the
database. To do that, you will need to run the following commands,
# Make sure you are in the 'app' directory. Adjust the 'cd' path to
# match your set-up.
cd ~/www/midpoint/app/
python3 build-database.py
This will clear the database if one is already set-up and then build a
new one.