|
|
@ -14,8 +14,241 @@ information on setting the repository up. |
|
|
|
python dailies-overlayed.py |
|
|
|
python dailies-overlayed.py |
|
|
|
#+end_src |
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Design/Trade-Off Notes and Decisions |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. This script processes the data produced by both Light Meters. |
|
|
|
|
|
|
|
2. I did not add a legend to the chart because there are too many lines plotted |
|
|
|
|
|
|
|
and it makes the chart difficult to read -- legend either takes up too much |
|
|
|
|
|
|
|
space or it simply over-runs off the screen. |
|
|
|
|
|
|
|
3. The values on the x-axis do not represent a timestamp. Each reading has an |
|
|
|
|
|
|
|
Id. number assigned to it. This is because of the inconsistencies between |
|
|
|
|
|
|
|
each day's data. The timestamps between each do not align with each other |
|
|
|
|
|
|
|
(messy data) so assigning each reading an Id. number was the easiest solution |
|
|
|
|
|
|
|
to get the scripts to run. |
|
|
|
|
|
|
|
1. The general pattern of the reading were the priority here, not the |
|
|
|
|
|
|
|
accuracy of when the readings were recorded. |
|
|
|
|
|
|
|
4. The exhibition ran until the 1^st August 2021 but that was a Sunday and no |
|
|
|
|
|
|
|
one, in Ritherdon, worked that day -- so there were no reading taken (I.E. no |
|
|
|
|
|
|
|
data to process). |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Files Used |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+begin_src shell :results code |
|
|
|
|
|
|
|
# NOTE: I have altered the list below based on the output produced by the |
|
|
|
|
|
|
|
# following line... |
|
|
|
|
|
|
|
tree -L 1 ../data/light-meter-* |
|
|
|
|
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+RESULTS: |
|
|
|
|
|
|
|
#+begin_src shell |
|
|
|
|
|
|
|
../data/light-meter-1 |
|
|
|
|
|
|
|
├── 2021-06-13.csv |
|
|
|
|
|
|
|
├── 2021-06-14.csv |
|
|
|
|
|
|
|
├── 2021-06-15.csv |
|
|
|
|
|
|
|
├── 2021-06-16.csv |
|
|
|
|
|
|
|
├── 2021-06-17.csv |
|
|
|
|
|
|
|
├── 2021-06-18.csv |
|
|
|
|
|
|
|
├── 2021-06-19.csv |
|
|
|
|
|
|
|
├── 2021-06-20.csv |
|
|
|
|
|
|
|
├── 2021-06-21.csv |
|
|
|
|
|
|
|
├── 2021-06-22.csv |
|
|
|
|
|
|
|
├── 2021-06-23.csv |
|
|
|
|
|
|
|
├── 2021-06-24.csv |
|
|
|
|
|
|
|
├── 2021-06-25.csv |
|
|
|
|
|
|
|
├── 2021-06-26.csv |
|
|
|
|
|
|
|
├── 2021-06-27.csv |
|
|
|
|
|
|
|
├── 2021-06-28.csv |
|
|
|
|
|
|
|
├── 2021-06-29.csv |
|
|
|
|
|
|
|
├── 2021-06-30.csv |
|
|
|
|
|
|
|
├── 2021-07-01.csv |
|
|
|
|
|
|
|
├── 2021-07-02.csv |
|
|
|
|
|
|
|
├── 2021-07-03.csv |
|
|
|
|
|
|
|
├── 2021-07-04.csv |
|
|
|
|
|
|
|
├── 2021-07-05.csv |
|
|
|
|
|
|
|
├── 2021-07-06.csv |
|
|
|
|
|
|
|
├── 2021-07-07.csv |
|
|
|
|
|
|
|
├── 2021-07-08.csv |
|
|
|
|
|
|
|
├── 2021-07-09.csv |
|
|
|
|
|
|
|
├── 2021-07-10.csv |
|
|
|
|
|
|
|
├── 2021-07-11.csv |
|
|
|
|
|
|
|
├── 2021-07-12.csv |
|
|
|
|
|
|
|
├── 2021-07-13.csv |
|
|
|
|
|
|
|
├── 2021-07-14.csv |
|
|
|
|
|
|
|
├── 2021-07-15.csv |
|
|
|
|
|
|
|
├── 2021-07-16.csv |
|
|
|
|
|
|
|
├── 2021-07-17.csv |
|
|
|
|
|
|
|
├── 2021-07-18.csv |
|
|
|
|
|
|
|
├── 2021-07-19.csv |
|
|
|
|
|
|
|
├── 2021-07-20.csv |
|
|
|
|
|
|
|
├── 2021-07-21.csv |
|
|
|
|
|
|
|
├── 2021-07-22.csv |
|
|
|
|
|
|
|
├── 2021-07-23.csv |
|
|
|
|
|
|
|
├── 2021-07-24.csv |
|
|
|
|
|
|
|
├── 2021-07-25.csv |
|
|
|
|
|
|
|
├── 2021-07-26.csv |
|
|
|
|
|
|
|
├── 2021-07-27.csv |
|
|
|
|
|
|
|
├── 2021-07-28.csv |
|
|
|
|
|
|
|
├── 2021-07-29.csv |
|
|
|
|
|
|
|
└── 2021-07-30.csv |
|
|
|
|
|
|
|
../data/light-meter-2 |
|
|
|
|
|
|
|
├── 2021-06-13.csv |
|
|
|
|
|
|
|
├── 2021-06-14.csv |
|
|
|
|
|
|
|
├── 2021-06-15.csv |
|
|
|
|
|
|
|
├── 2021-06-16.csv |
|
|
|
|
|
|
|
├── 2021-06-17.csv |
|
|
|
|
|
|
|
├── 2021-06-18.csv |
|
|
|
|
|
|
|
├── 2021-06-19.csv |
|
|
|
|
|
|
|
├── 2021-06-20.csv |
|
|
|
|
|
|
|
├── 2021-06-21.csv |
|
|
|
|
|
|
|
├── 2021-06-22.csv |
|
|
|
|
|
|
|
├── 2021-06-23.csv |
|
|
|
|
|
|
|
├── 2021-06-24.csv |
|
|
|
|
|
|
|
├── 2021-06-25.csv |
|
|
|
|
|
|
|
├── 2021-06-26.csv |
|
|
|
|
|
|
|
├── 2021-06-27.csv |
|
|
|
|
|
|
|
├── 2021-06-28.csv |
|
|
|
|
|
|
|
├── 2021-06-29.csv |
|
|
|
|
|
|
|
├── 2021-06-30.csv |
|
|
|
|
|
|
|
├── 2021-07-01.csv |
|
|
|
|
|
|
|
├── 2021-07-02.csv |
|
|
|
|
|
|
|
├── 2021-07-03.csv |
|
|
|
|
|
|
|
├── 2021-07-04.csv |
|
|
|
|
|
|
|
├── 2021-07-05.csv |
|
|
|
|
|
|
|
├── 2021-07-06.csv |
|
|
|
|
|
|
|
├── 2021-07-07.csv |
|
|
|
|
|
|
|
├── 2021-07-08.csv |
|
|
|
|
|
|
|
├── 2021-07-09.csv |
|
|
|
|
|
|
|
├── 2021-07-10.csv |
|
|
|
|
|
|
|
├── 2021-07-11.csv |
|
|
|
|
|
|
|
├── 2021-07-12.csv |
|
|
|
|
|
|
|
├── 2021-07-13.csv |
|
|
|
|
|
|
|
├── 2021-07-14.csv |
|
|
|
|
|
|
|
├── 2021-07-15.csv |
|
|
|
|
|
|
|
├── 2021-07-16.csv |
|
|
|
|
|
|
|
├── 2021-07-17.csv |
|
|
|
|
|
|
|
├── 2021-07-18.csv |
|
|
|
|
|
|
|
├── 2021-07-19.csv |
|
|
|
|
|
|
|
├── 2021-07-20.csv |
|
|
|
|
|
|
|
├── 2021-07-21.csv |
|
|
|
|
|
|
|
├── 2021-07-22.csv |
|
|
|
|
|
|
|
├── 2021-07-23.csv |
|
|
|
|
|
|
|
├── 2021-07-24.csv |
|
|
|
|
|
|
|
├── 2021-07-25.csv |
|
|
|
|
|
|
|
├── 2021-07-26.csv |
|
|
|
|
|
|
|
├── 2021-07-27.csv |
|
|
|
|
|
|
|
├── 2021-07-28.csv |
|
|
|
|
|
|
|
├── 2021-07-29.csv |
|
|
|
|
|
|
|
└── 2021-07-30.csv |
|
|
|
|
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Files Produced |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+begin_src shell :results code |
|
|
|
|
|
|
|
ls ../output/lm*-dailies-overlayed.html |
|
|
|
|
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+RESULTS: |
|
|
|
|
|
|
|
#+begin_src shell |
|
|
|
|
|
|
|
../output/lm1-dailies-overlayed.html |
|
|
|
|
|
|
|
../output/lm2-dailies-overlayed.html |
|
|
|
|
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
* What the Script Does |
|
|
|
* What the Script Does |
|
|
|
|
|
|
|
|
|
|
|
This needs to be written... |
|
|
|
#+begin_quote |
|
|
|
|
|
|
|
The charts produced by this script are not accurate but *useful*. |
|
|
|
|
|
|
|
#+end_quote |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This script produces two charts. It does this by taking all the daily reading |
|
|
|
|
|
|
|
for both Light Meters (the CSV files listed above) and overlays them on top of |
|
|
|
|
|
|
|
each other. Each chart represents the readings from a given Light |
|
|
|
|
|
|
|
Meter. =output/lm1-dailies-overlayed.html= show all the readings for Light Meter 1 |
|
|
|
|
|
|
|
and =/output/lm2-dailies-overlayed.html= show the readings for Light Meter 2. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Each line on both charts represents all the readings for a given day. *The x-axis |
|
|
|
|
|
|
|
values do not denote a timestamp, though.* What this means is the lines in the |
|
|
|
|
|
|
|
charts do not include the time data. They only illustrate the light levels |
|
|
|
|
|
|
|
recorded in sequential order -- the time is took between each change is |
|
|
|
|
|
|
|
missing. This is why these charts look different when compared to the charts of |
|
|
|
|
|
|
|
each individual day. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Below is an example of what the data looks like when the script is first |
|
|
|
|
|
|
|
run. The data is from =data/light-meter-1/2021-06-19.csv=. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| time | reading | |
|
|
|
|
|
|
|
|----------------------------+---------| |
|
|
|
|
|
|
|
| 2021-06-19 07:03:11.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:12.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:13.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:14.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:15.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:16.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:17.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:18.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:19.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:20.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:21.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:23.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:24.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:25.000000 | 13 | |
|
|
|
|
|
|
|
| MORE READINGS HERE... | ... | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:47.000000 | 3 | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:49.000000 | 3 | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:52.000000 | 3 | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:54.000000 | 3 | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:57.000000 | 3 | |
|
|
|
|
|
|
|
| 2021-06-19 15:59:59.000000 | 3 | |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Because each day has different ~time~ values, it makes is difficult to map the |
|
|
|
|
|
|
|
data from each file on to a single x-axis. I decided to take the easy route and |
|
|
|
|
|
|
|
map each time stamp to a sequential number. In other words, |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| time | reading | |
|
|
|
|
|
|
|
|----------------------------+---------| |
|
|
|
|
|
|
|
| 2021-06-19 07:03:11.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:12.000000 | 13 | |
|
|
|
|
|
|
|
| 2021-06-19 07:03:13.000000 | 13 | |
|
|
|
|
|
|
|
| MORE READINGS HERE... | ... | |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
becomes, |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| time | reading | |
|
|
|
|
|
|
|
|--------------------------------------+---------| |
|
|
|
|
|
|
|
| 1 | 13 | |
|
|
|
|
|
|
|
| 2 | 13 | |
|
|
|
|
|
|
|
| 3 | 13 | |
|
|
|
|
|
|
|
| SEQUENCE CONTINUES TO END OF FILE... | | |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
You will see that in the following section of code in the =dailies-overlayed.py= |
|
|
|
|
|
|
|
script. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+begin_src python |
|
|
|
|
|
|
|
x_raw_vals = [] |
|
|
|
|
|
|
|
y_raw_vals = [] |
|
|
|
|
|
|
|
counter = 0 |
|
|
|
|
|
|
|
for row in file_data : |
|
|
|
|
|
|
|
x_raw_vals.append(counter) # The timestamp is replace here... |
|
|
|
|
|
|
|
counter += 1 |
|
|
|
|
|
|
|
y_raw_vals.append(row[1]) |
|
|
|
|
|
|
|
#+end_src |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Below is an example of how the charts differ when you remove the timestamped |
|
|
|
|
|
|
|
data from the x-axis. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[file:../assets/x-axis-timestamped-example.png]] |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[file:../assets/x-axis-sequential-id-example.png]] |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Both charts above have the same y-axis values but the scale of the x-axis is |
|
|
|
|
|
|
|
different so the lines appear different. The /same general pattern/ emerges, |
|
|
|
|
|
|
|
though, and that is the piece of information we/Nicola/I want in this instance. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
I repeat again, |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+begin_quote |
|
|
|
|
|
|
|
The charts produced by this script are not accurate but *useful*. |
|
|
|
|
|
|
|
#+end_quote |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Examples/Screenshots |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[file:../assets/lm1-overlayed.png]] |
|
|
|