2021-11-05 snapshot.

commit eda360a068
Merge: 0b9b295 071c8cf
Author: Craig Oates <craig@craigoates.net>
Date:   Fri Nov 5 18:24:33 2021 +0000

    Merge branch 'unstable' into stable

commit 071c8cf5f3
Merge: 1534c04 56e05dc
Author: Craig Oates <craig@craigoates.net>
Date:   Fri Nov 5 18:16:43 2021 +0000

    Merge branch 'unstable' of git.abbether.net:return-to-ritherdon/light-meter into unstable

commit 1534c04635
Author: Craig Oates <craig@craigoates.net>
Date:   Fri Nov 5 18:15:45 2021 +0000

    update README.

commit 56e05dceaf
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Fri Feb 5 16:54:31 2021 +0000

    use session to persist REST calls and expand exception handling.

    Previously, the light-meter was creating a new HTTP request to send every new reading. This now
    uses a persistant session to reduce the amount of requests made to the server -- and reduce low
    on the system.

    The extra exceptions listed and expanded on are mostly for testing. At the time of writing, the
    system as a whole is hanging at various parts throughout the day and the intention is to monitor
    the Light Meter to see if any of the new exceptions are the cause (or contributing) to the
    hanging. The current trains-of-thought at the minute are too many requests (system load too
    high) and requests not timing out properly.

commit b4571b0c1e
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Wed Jan 27 21:59:51 2021 +0000

    create a Systemd service file.

    This file is used to start the program at startup. It, also, restarts the
    program if it crashes or throws an exception. The usual scenario which
    causes the program to crash is the 'max. requests exceeded' exception in
    the 'requests' module (Python). When that happens, this 'restart' service
    has a 60 seconds wait period -- which acts as a 'backoff' period.

    You will need to copy the .service file to the appropriate systemd
    location, 'daemon-reload' systemctl, enable the service and start the
    service. You will need to look-up how to do that -- this info. is beyond
    the scope of this commmit.

commit 0b9b295036
Merge: 814fbdf b7140cb
Author: Craig Oates <craig@craigoates.net>
Date:   Wed Jan 13 13:08:23 2021 +0000

    Merge branch 'unstable' of return-to-ritherdon/light-meter into stable

commit b7140cbf2b
Author: Craig Oates <craig@craigoates.net>
Date:   Wed Jan 13 12:59:44 2021 +0000

    change info. about cronjob in README.

    This is a minor change but I thought it was best to document it.

    The factory typically shutdowns at 16:00  but for some reason I thought it was 18:00. Because of this, 'factory1' (the light-meter used for testing) was not shutting down properly. The factory's power was cut at the end of the day (again typically 16:00 but not always) and the Pi would have a dirty power-down. This, also, meant the server and its opposite relay [return-to-ritherdon/relay] would think it was still on but sending no new readings.

    All of the above has no great effect of the **note** left in the README but I thought I should change it on the off-chance someone copies that line into a part of the system.

commit 46b8d53cc4
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Wed Jan 13 11:58:27 2021 +0000

    remove merge conflict artefacts.

commit 2ad344846b
Merge: 39cf8bc 6e4e425
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Wed Jan 13 11:51:02 2021 +0000

    Merge remote-tracking branch 'origin/fact2temp' into unstable

commit 39cf8bc302
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Wed Jan 13 11:22:47 2021 +0000

    update URL used for RESTAPI calls.

    I have given the 'mid-point' server a domain name to reduce the need to manually update the IP address if/when the
    server needs to be restarted (for whatever reason). The changes here include the new URL.

commit 6e4e4254cf
Author: Craig Oates <craig@craigoates.net>
Date:   Wed Dec 16 17:18:07 2020 +0000

    extend URL list for adding new light readings.

    The base URL uses ritherdon.abbether.net now, in the main part of the
    python code. I have added checks to see which device is running the
    code and forms a complete URL for the API call.

    The way the URL is formed for the API call is not the most elegant
    solution but is was quick to write and the code has a limited
    life-span. It will not require any further modification when it goes
    live. So, the speed it took to write it was a good trade-off in my opinion.

commit 7a3e3df07a
Author: Craig Oates <craig@craigoates.net>
Date:   Wed Dec 16 16:53:19 2020 +0000

    change URL to ritherdon.abbether.net

    The URL used here is a domain name and not just an IP address. This
    should reduce the need to update the IP address in the code if the
    server, currently hosted on AWS at time of writing, needs to be
    restarted. The servers on AWS do not keep the same IP address if you
    stop and start them.

    The abbether.net domain is a personal one and used as a way to reduce
    project costs. I did not use this domain when I first wrote the code
    here because I did not own it. Long story short, the Covid-19 malarkey
    meant the project was put on hold and I ended up owning abbether.net
    during the first lock-down. The decision to use the domain is one of
    convince my end. I did not want to keep updating the code manually if
    the server on AWS was stopping and starting.

commit 9532df8137
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Sun Jan 12 19:22:56 2020 +0000

    adjust shutdown call in shutdown.sh.

commit a75cf4083f
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Jan 12 02:00:25 2020 +0000

    add residency overview image to readme.

commit b0d6474809
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Jan 12 01:39:48 2020 +0000

    comment out testing/debugging print states in cli_meter.

commit 67bef91b14
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Jan 12 01:34:03 2020 +0000

    fix typo's in readme.

commit beccebc3ec
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Jan 12 01:15:14 2020 +0000

    add main-proj-architecture image (for readme).

commit 22bd78800c
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Jan 12 00:18:10 2020 +0000

    update images in readme.

commit 234dffe4c0
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 23:40:42 2020 +0000

    make edit to readme structure.

commit 9ccdc91778
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 23:34:47 2020 +0000

    add images for readme.

commit d99e04c87d
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 23:12:03 2020 +0000

    add hardware section to readme.

commit 445adcc53a
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 22:04:00 2020 +0000

    edit the readme.

commit fa98024e0f
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 21:33:38 2020 +0000

    update readme.

commit 8e98fe2fac
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Jan 11 18:53:23 2020 +0000

    add comments to startup, shutdown and cli-meter.

commit 0eaea3b798
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Sat Jan 11 01:11:22 2020 +0000

    refactor timers in scripts.

commit 9027a3bcb1
Author: rtrp@factory1 <rtrp@factory1.com>
Date:   Fri Jan 10 21:02:05 2020 +0000

    create a make-logs script.

commit 5f264e7f60
Author: Craig <craig@craigoates.net>
Date:   Fri Jan 10 01:28:21 2020 +0000

    add times to startup and shutdown scripts.

commit f0f4776a9b
Author: Craig <craig@craigoates.net>
Date:   Fri Jan 10 00:07:26 2020 +0000

    add start-up and shutdown scripts.

commit c5da9d5918
Author: Craig <craig@craigoates.net>
Date:   Sun Jan 5 19:45:45 2020 +0000

    add keyboard interrup exception.

commit 9ef10aef16
Author: Craig <craig@craigoates.net>
Date:   Sun Jan 5 19:15:40 2020 +0000

    port 'cli' code from light_meter to cli_meter.

commit f89e7b3923
Author: Craig Oates <craig@craigoates.net>
Date:   Sun Dec 15 00:12:59 2019 +0000

    add cli-meter.py

commit 32e5b464c6
Author: Craig Oates <craig@craigoates.net>
Date:   Sat Dec 14 23:52:28 2019 +0000

    port over code from roaming-light project.

README.md

@@ -1,3 +1,13 @@
-# light-meter
+# Return to Riterdon: Light Meter
 
-A Python based project which measures the light levels of an environment and forwards those measurements on to a server (for documentation purposes). This project is one of several which forms the software-side of the "Return to Ritherdon" project.
+Light Meter is a Python 3 based project for taking light readings (in
+the welding booths in Ritherdon) and sending them to the 'Midpoint'
+project for storage. Light Meter is one of three projects which
+combine to make the 'Personal Flash in Real-Time' artwork -- which is
+one of several artworks created as part of the 'Return to Ritherdon'
+project. For more information on this repository and 'Return to
+Ritherdon', please use the links below:
+
+- [Return to Ritherdon Overview](https://git.abbether.net/return-to-ritherdon/rtr-docs)
+- [Light Meter Documentation](https://git.abbether.net/return-to-ritherdon/rtr-docs/src/branch/master/light-meter/rtr-light-meter.md)
+  

cli_meter.py

@@ -0,0 +1,131 @@
+#!/usr/bin/python3
+
+"""
+C.L.I. Meter
+======================================================================
+I wrote this script with the intention of using it as one part within
+a single cron-job. The cron-job begins with "startup.sh" and that,
+in-turn, will call this script when it is ready to do so.
+
+This scrip takes a reading of the current light-level in the room,
+using the Raspberry Pi (and the attached light-meter attached) it is
+running on. From there, it sends a J.S.O.N. object to the server
+(specified at "api_url"). The process is then repeated until it is
+killed manually or via "shutdown.sh" (as a cron-job most likely).
+
+You can use this script on Raspberry Pis without a G.U.I. because it
+is all C.L.I. based. If you prefer something more graphical, you can
+use "light_meter.py" -- which is part of this repository at time of
+writing.
+
+To end this script manually, use the usual "Ctrl-c" keyboard
+interrupt -- or something like Htop... doesn't matter.
+"""
+
+import json
+import RPi.GPIO as GPIO
+import time
+import math
+import requests
+from datetime import datetime
+import platform
+# import pdb # For testing.
+
+# Using BCM (Broadcom) names when referencing the GPIO pins.
+GPIO.setmode(GPIO.BCM)
+GPIO.setwarnings(True)
+
+a_pin = 18  # Charges the capacitor.
+b_pin = 23  # Discharges the capacitor.
+
+device_id = platform.node()  # For servers logs.
+session = requests.Session() # Persist connection for REST calls.
+
+def get_api_url():
+    # Yes, I know I could do this better. Stop moaning.
+    if (device_id == "factory1"):
+        return "http://ritherdon.abbether.net/api/readings/add/1"
+    elif (device_id == "factory2"):
+        return "http://ritherdon.abbether.net/api/readings/add/2"
+
+api_url = get_api_url()
+
+def discharge():
+    GPIO.setup(a_pin, GPIO.IN)
+    GPIO.setup(b_pin, GPIO.OUT)
+    GPIO.output(b_pin, False)
+    time.sleep(0.01)  # 0.01 -- initial/default value.
+
+def charge_time():
+    GPIO.setup(b_pin, GPIO.IN)
+    GPIO.setup(a_pin, GPIO.OUT)
+    GPIO.output(a_pin, True)
+    t1 = time.time()
+    while not GPIO.input(b_pin):
+        pass
+    t2 = time.time()
+    return (t2 - t1) * 1000000
+
+def analog_read():
+    discharge()
+    return charge_time()
+
+def read_resistance():
+    n = 20
+    total = 0
+    for i in range(1, n):
+        total = total + analog_read()
+    reading = total / float(n)
+    resistance = reading * 6.05 - 939
+    return resistance
+
+def light_from_r(R):
+    return math.log(1000000.0/R) * 10.0
+
+def get_timestamp():
+    return datetime.now().strftime(("%Y-%m-%d %H:%M:%S"))
+
+def push_reading(lvalue):
+    time = get_timestamp()
+    headers = {"content-type": "application/json"}
+    payload = {"reading": int(lvalue), "time": time,
+               "token": "QWERTYuiopasdfghjklzxcvbnm_1234567890"}
+    print(payload) # For testing.
+    # res = requests.post(api_url, data=json.dumps(payload), headers=headers)
+    res = session.post(api_url, data=json.dumps(payload), headers=headers)
+
+def update_reading():
+    light = light_from_r(read_resistance())
+    reading_str = "{:.0f}".format(light)
+    # print(reading_str) # For testing.
+    push_reading(light)
+
+
+def main():
+    try:
+        while True:
+            # pdb.set_trace() # For testing.
+            update_reading()
+    except KeyboardInterrupt:
+        print("[INFO] KEYBOARD INTERRUPT: quitting program.")
+    except requests.exceptions.ConnectionError:
+        pause = 60
+        time.sleep(60)
+        print(f"[WARNING] MAX. REQUESTS EXCEEDED: Pausing requests for {pause} seconds...")
+        pass
+    except requests.exceptions.Timeout:
+        t_stamp = datetime.datetime.now()
+        print(f"[WARNING] TIMEOUT EXCEPTION: Request timed-out at {t_stamp}.")
+        time.sleep(60)
+        pass
+    except Exception as e:
+        print(f"[ERROR] GENERAL EXCEPTION: {e}")
+    finally:
+        print("[INFO] Terminating relay.py...")
+        print("[INFO] Cleaning up GPIO before closing...")
+        GPIO.cleanup()
+
+
+if __name__ == "__main__":
+    # time.sleep(60) # For testing/debugging.
+    main()

light-meter.service

@@ -0,0 +1,16 @@
+[Unit]
+Description=Light-Meter service for Return to Ritherdon project by Nicola Ellis
+After=network.service
+
+[Service]
+Type=simple
+ExecStart=bash /home/rtrp/repos/light-meter/startup.sh
+WorkingDirectory=/home/rtrp/repos/light-meter
+StandardOutput=inherit
+StandardError=inherit
+Restart=always
+RestartSec=60
+user=rtrp
+
+[Install]
+WantedBy=multi-user.target

light_meter.py

@@ -0,0 +1,129 @@
+# 07_light_meter.py
+# From the code for the Electronics Starter Kit for the Raspberry Pi by MonkMakes.com
+
+#from Tkinter import *
+from tkinter import *
+import json
+import RPi.GPIO as GPIO
+import time, math
+import requests # for pushing the readings to roaming-light-api
+from datetime import datetime
+
+# Configure the Pi to use the BCM (Broadcom) pin names, rather than the pin positions
+GPIO.setmode(GPIO.BCM)
+
+# This project uses a photoresistor, a component whose resistance varies with the light falling on it.
+# To measure its resistance, the code records the time it takes for a capacitor to fill  
+# when supplied by a current passing through the resistor. The lower the resistance the faster 
+# it fills up. 
+#
+# You can think of a capacitor as a tank of electricity, and as it fills with charge, the voltage
+# across it increases. We cannot measure that voltage directly, because the Raspberry Pi
+# does not have an analog to digital convertor (ADC or analog input). However, we can time how long it
+# takes for the capacitor to fill with charge to the extent that it gets above the 1.65V or so
+# that counts as being a high digital input. 
+# 
+# For more information on this technique take a look at: 
+# learn.adafruit.com/basic-resistor-sensor-reading-on-raspberry-pi
+# The code here is based on that in the Raspberry Pi Cookbook (Recipes 12.1 to 12.3)
+
+
+# Pin a charges the capacitor through a fixed 1k resistor and the thermistor in series
+# pin b discharges the capacitor through a fixed 1k resistor 
+a_pin = 18
+b_pin = 23
+
+# empty the capacitor ready to start filling it up
+def discharge():
+    GPIO.setup(a_pin, GPIO.IN)
+    GPIO.setup(b_pin, GPIO.OUT)
+    GPIO.output(b_pin, False)
+    time.sleep(0.01) # 0.01 -- initial value
+
+# return the time taken for the voltage on the capacitor to count as a digital input HIGH
+# than means around 1.65V
+def charge_time():
+    GPIO.setup(b_pin, GPIO.IN)
+    GPIO.setup(a_pin, GPIO.OUT)
+    GPIO.output(a_pin, True)
+    t1 = time.time()
+    while not GPIO.input(b_pin):
+        pass
+    t2 = time.time()
+    return (t2 - t1) * 1000000
+
+# Take an analog readin as the time taken to charge after first discharging the capacitor
+def analog_read():
+    discharge()
+    return charge_time()
+
+# Convert the time taken to charge the cpacitor into a value of resistance
+# To reduce errors, do it 100 times and take the average.
+def read_resistance():
+    n = 20
+    total = 0;
+    for i in range(1, n):
+        total = total + analog_read()
+    reading = total / float(n)
+    resistance = reading * 6.05 - 939
+    return resistance
+
+def light_from_r(R):
+    # Log the reading to compress the range
+    return math.log(1000000.0/R) * 10.0
+
+# Gets the time and date (at the time of the reading)
+def get_timestamp():
+    return datetime.now().strftime(("%Y-%m-%d %H:%M:%S"))
+
+# Pushes the reading to roaming-light-api
+def push_reading(lvalue):
+    act = False
+    if lvalue > 20:
+        act = True
+    time = get_timestamp()
+    headers = {"content-type": "application/json"}
+#    payload = { "active": act, "light-reading": int(lvalue), "timestamp": time }
+    payload = { "reading": int(lvalue), "time": time}
+#    res = requests.post("http://192.168.1.228:5000/api/readings/add/1", data=json.dumps(payload), headers=headers)
+    res = requests.post("http://ritherdon.abbether.net/api/readings/add/1", data=json.dumps(payload), headers=headers)
+#    print(int(lvalue))
+#    print(res.status_code)
+    print(payload)
+
+# group together all of the GUI code into a class called App
+class App:
+
+    # this function gets called when the app is created
+    def __init__(self, master):
+        self.master = master
+        frame = Frame(master, bg="black")
+        frame.pack(expand=1)
+        label = Label(frame, text='Light', fg="white", bg="black", font=("Helvetica", 32))
+        label.grid(row=0)
+        self.reading_label = Label(frame, text='12.34', bg="black", fg="white", font=("Helvetica", 110))
+        self.reading_label.grid(row=1)
+        self.update_reading()
+
+    # Update the reading
+    def update_reading(self):
+        light = light_from_r(read_resistance())
+        reading_str = "{:.0f}".format(light)
+        self.reading_label.configure(text=reading_str)
+        self.master.after(200, self.update_reading)
+        push_reading(light)
+
+# Set the GUI running, give the window a title, size and position
+root = Tk()
+# Uncomment when app. is ready for deployment
+#root.attributes("-fullscreen", True)
+root.attributes("-zoomed", True)
+root.configure(background="black")
+root.wm_title('Light Meter')
+app = App(root)
+root.geometry("400x300+0+0")
+try:
+    root.mainloop()
+finally:
+    print("Cleaning up")
+    GPIO.cleanup()

make-log-files.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+# This script creates the log files and the fold they will reside in.
+# This script must be executed before you set-up the start-up and
+# shutdown cron-jobs on the system.
+
+mkdir ~/logs/
+touch ~/logs/startup-logs.txt
+touch ~/logs/shutdown-logs.txt

shutdown.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+
+# Shutdown Script
+# ====================================================================
+# I wrote this script with the intention of using it as part of a
+# cron-job. Before you set up the cron-job, you must make sure you
+# have either ran the "make-log-files.sh" script or created the
+# appropriate log file and folder at the location specified at
+# "logFile" below.
+# As an aside, I usually set-up as alias for "shutdown" called,
+# "powerdown". When I enter "powerdown" into the terminal, this script
+# should ran and then the "shutdown" command is ran at the end. The
+# "sleep" part is to give "curl" a bit of a buffer to finish its task.
+# I have found it sometimes fails otherwise.
+
+logDate=$(date '+%Y-%m-%dT%TZ')
+logFile="/home/rtrp/logs/shutdown-logs.txt"
+mainURL="http://ritherdon.abbether.net/api/status/update" # Make sure this is valid.
+
+getApiUrl () {
+    case $HOSTNAME in
+    	(factory1) apiURL="${mainURL}/1";;
+    	(factory2) apiURL="${mainURL}/2";;
+    	(factory3) apiURL="${mainURL}/3";;
+    	(gallery1) apiURL="${mainURL}/4";;
+    	(gallery2) apiURL="${mainURL}/5";;
+    	(gallery3) apiURL="${mainURL}/6";;
+    esac
+}
+
+logStatusChange () {
+    cat << EOF >> $logFile
+$logDate
+EOF
+}
+
+logStatusChange
+getApiUrl
+
+curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/html' -d '{"status": "off", "time": "'${logDate}'", "token": "QWERTYuiopasdfghjklzxcvbnm_1234567890"}' "${apiURL}"
+
+/sbin/shutdown -h now

startup.sh

@@ -0,0 +1,51 @@
+#!/bin/bash  
+
+# Startup Script
+# ====================================================================
+# I wrote this script with the intention of using it to start a
+# cron-job -- when the Raspberry Pi is turned on. Before you set the
+# cron-job up, you must make sure you have either ran the
+# "make-log-files.sh" script or created the appropriate log file and
+# folder at the location specified at "logFile" below.
+# ====
+# Please note: This script calls another (Python) script at the end of
+# it. The code in the Python script (cli_meter.py) is an infinite-loop
+# so you will need to kill it manually or turn-off the  Raspberry Pi.
+# ====
+# I put the "sleep 60" call at the start to reduce any errors
+# occurring because a part of the system (I.E. curl) has not finished
+# loading. 60 seconds is a little excessive but I wrote this script
+# with the expectation of it running on an unmanned Raspberry Pi. So,
+# reliable/consistent behaviour is my preference over "fast start-up
+# times".
+
+
+# sleep 60
+
+logDate=$(date '+%Y-%m-%dT%TZ')
+logFile="/home/rtrp/logs/startup-logs.txt"
+mainURL="http://ritherdon.abbether.net/api/status/update" # Make sure this is valid.
+
+getApiUrl () {
+    case $HOSTNAME in
+    	(factory1) apiURL="${mainURL}/1";;
+    	(factory2) apiURL="${mainURL}/2";;
+    	(factory3) apiURL="${mainURL}/3";;
+    	(gallery1) apiURL="${mainURL}/4";;
+    	(gallery2) apiURL="${mainURL}/5";;
+    	(gallery3) apiURL="${mainURL}/6";;
+    esac
+}
+
+logStatusChange () {
+    cat << EOF >> $logFile
+$logDate
+EOF
+}
+
+logStatusChange
+getApiUrl
+
+curl -S -X POST --header 'Content-Type: application/json' --header 'Accept: text/html' -d '{"status": "on", "time": "'${logDate}'", "token": "QWERTYuiopasdfghjklzxcvbnm_1234567890"}' "${apiURL}"
+
+python3 /home/rtrp/repos/light-meter/cli_meter.py