* Overview of =dailies-overlayed.py= Script I have assumed you have ran ~./separator.sh~ and ~./totalilator.sh~, created a virtual-environment (venv) (~python3 -m venv venv~) and installed the dependencies (via ~pip -r install requirements.txt~). See the project's [[file:../README.org][README]] for more information on setting the repository up. #+begin_src shell cd # Activate the virtual-environment if you haven't... source venv/bin/activate # Run the script... python dailies-overlayed.py #+end_src * Design/Trade-Off Notes and Decisions 1. This script processes the data produced by both Light Meters. 2. I did not add a legend to the chart because there are too many lines plotted and it makes the chart difficult to read -- legend either takes up too much space or it simply over-runs off the screen. 3. The values on the x-axis do not represent a timestamp. Each reading has an Id. number assigned to it. This is because of the inconsistencies between each day's data. The timestamps between each do not align with each other (messy data) so assigning each reading an Id. number was the easiest solution to get the scripts to run. 1. The general pattern of the reading were the priority here, not the accuracy of when the readings were recorded. 4. The exhibition ran until the 1^st August 2021 but that was a Sunday and no one, in Ritherdon, worked that day -- so there were no reading taken (I.E. no data to process). * Files Used #+begin_src shell :results code # NOTE: I have altered the list below based on the output produced by the # following line... tree -L 1 ../data/light-meter-* #+end_src #+RESULTS: #+begin_src shell ../data/light-meter-1 ├── 2021-06-13.csv ├── 2021-06-14.csv ├── 2021-06-15.csv ├── 2021-06-16.csv ├── 2021-06-17.csv ├── 2021-06-18.csv ├── 2021-06-19.csv ├── 2021-06-20.csv ├── 2021-06-21.csv ├── 2021-06-22.csv ├── 2021-06-23.csv ├── 2021-06-24.csv ├── 2021-06-25.csv ├── 2021-06-26.csv ├── 2021-06-27.csv ├── 2021-06-28.csv ├── 2021-06-29.csv ├── 2021-06-30.csv ├── 2021-07-01.csv ├── 2021-07-02.csv ├── 2021-07-03.csv ├── 2021-07-04.csv ├── 2021-07-05.csv ├── 2021-07-06.csv ├── 2021-07-07.csv ├── 2021-07-08.csv ├── 2021-07-09.csv ├── 2021-07-10.csv ├── 2021-07-11.csv ├── 2021-07-12.csv ├── 2021-07-13.csv ├── 2021-07-14.csv ├── 2021-07-15.csv ├── 2021-07-16.csv ├── 2021-07-17.csv ├── 2021-07-18.csv ├── 2021-07-19.csv ├── 2021-07-20.csv ├── 2021-07-21.csv ├── 2021-07-22.csv ├── 2021-07-23.csv ├── 2021-07-24.csv ├── 2021-07-25.csv ├── 2021-07-26.csv ├── 2021-07-27.csv ├── 2021-07-28.csv ├── 2021-07-29.csv └── 2021-07-30.csv ../data/light-meter-2 ├── 2021-06-13.csv ├── 2021-06-14.csv ├── 2021-06-15.csv ├── 2021-06-16.csv ├── 2021-06-17.csv ├── 2021-06-18.csv ├── 2021-06-19.csv ├── 2021-06-20.csv ├── 2021-06-21.csv ├── 2021-06-22.csv ├── 2021-06-23.csv ├── 2021-06-24.csv ├── 2021-06-25.csv ├── 2021-06-26.csv ├── 2021-06-27.csv ├── 2021-06-28.csv ├── 2021-06-29.csv ├── 2021-06-30.csv ├── 2021-07-01.csv ├── 2021-07-02.csv ├── 2021-07-03.csv ├── 2021-07-04.csv ├── 2021-07-05.csv ├── 2021-07-06.csv ├── 2021-07-07.csv ├── 2021-07-08.csv ├── 2021-07-09.csv ├── 2021-07-10.csv ├── 2021-07-11.csv ├── 2021-07-12.csv ├── 2021-07-13.csv ├── 2021-07-14.csv ├── 2021-07-15.csv ├── 2021-07-16.csv ├── 2021-07-17.csv ├── 2021-07-18.csv ├── 2021-07-19.csv ├── 2021-07-20.csv ├── 2021-07-21.csv ├── 2021-07-22.csv ├── 2021-07-23.csv ├── 2021-07-24.csv ├── 2021-07-25.csv ├── 2021-07-26.csv ├── 2021-07-27.csv ├── 2021-07-28.csv ├── 2021-07-29.csv └── 2021-07-30.csv #+end_src * Files Produced #+begin_src shell :results code ls ../output/lm*-dailies-overlayed.html #+end_src #+RESULTS: #+begin_src shell ../output/lm1-dailies-overlayed.html ../output/lm2-dailies-overlayed.html #+end_src * What the Script Does #+begin_quote The charts produced by this script are not accurate but *useful*. #+end_quote This script produces two charts. It does this by taking all the daily reading for both Light Meters (the CSV files listed above) and overlays them on top of each other. Each chart represents the readings from a given Light Meter. =output/lm1-dailies-overlayed.html= show all the readings for Light Meter 1 and =/output/lm2-dailies-overlayed.html= show the readings for Light Meter 2. Each line on both charts represents all the readings for a given day. *The x-axis values do not denote a timestamp, though.* What this means is the lines in the charts do not include the time data. They only illustrate the light levels recorded in sequential order -- the time is took between each change is missing. This is why these charts look different when compared to the charts of each individual day. Below is an example of what the data looks like when the script is first run. The data is from =data/light-meter-1/2021-06-19.csv=. | time | reading | |----------------------------+---------| | 2021-06-19 07:03:11.000000 | 13 | | 2021-06-19 07:03:12.000000 | 13 | | 2021-06-19 07:03:13.000000 | 13 | | 2021-06-19 07:03:14.000000 | 13 | | 2021-06-19 07:03:15.000000 | 13 | | 2021-06-19 07:03:16.000000 | 13 | | 2021-06-19 07:03:17.000000 | 13 | | 2021-06-19 07:03:18.000000 | 13 | | 2021-06-19 07:03:19.000000 | 13 | | 2021-06-19 07:03:20.000000 | 13 | | 2021-06-19 07:03:21.000000 | 13 | | 2021-06-19 07:03:23.000000 | 13 | | 2021-06-19 07:03:24.000000 | 13 | | 2021-06-19 07:03:25.000000 | 13 | | MORE READINGS HERE... | ... | | 2021-06-19 15:59:47.000000 | 3 | | 2021-06-19 15:59:49.000000 | 3 | | 2021-06-19 15:59:52.000000 | 3 | | 2021-06-19 15:59:54.000000 | 3 | | 2021-06-19 15:59:57.000000 | 3 | | 2021-06-19 15:59:59.000000 | 3 | Because each day has different ~time~ values, it makes is difficult to map the data from each file on to a single x-axis. I decided to take the easy route and map each time stamp to a sequential number. In other words, | time | reading | |----------------------------+---------| | 2021-06-19 07:03:11.000000 | 13 | | 2021-06-19 07:03:12.000000 | 13 | | 2021-06-19 07:03:13.000000 | 13 | | MORE READINGS HERE... | ... | becomes, | time | reading | |--------------------------------------+---------| | 1 | 13 | | 2 | 13 | | 3 | 13 | | SEQUENCE CONTINUES TO END OF FILE... | | You will see that in the following section of code in the =dailies-overlayed.py= script. #+begin_src python x_raw_vals = [] y_raw_vals = [] counter = 0 for row in file_data : x_raw_vals.append(counter) # The timestamp is replace here... counter += 1 y_raw_vals.append(row[1]) #+end_src Below is an example of how the charts differ when you remove the timestamped data from the x-axis. [[file:../assets/x-axis-timestamped-example.png]] [[file:../assets/x-axis-sequential-id-example.png]] Both charts above have the same y-axis values but the scale of the x-axis is different so the lines appear different. The /same general pattern/ emerges, though, and that is the piece of information we/Nicola/I want in this instance. I repeat again, #+begin_quote The charts produced by this script are not accurate but *useful*. #+end_quote * Examples/Screenshots [[file:../assets/lm1-overlayed.png]]