* Overview of =daily-breakdowns.py= Script

I have assumed you have ran ~./separator.sh~ and ~./totalilator.sh~, created a
virtual-environment (venv) (~python3 -m venv venv~) and installed the dependencies
(via ~pip -r install requirements.txt~). See the project's [[file:../README.org][README]] for more
information on setting the repository up.

#+begin_src shell
  cd <PATH TO RITHERDON-CHARTS>
  # Activate the virtual-environment if you haven't...
  source venv/bin/activate

  # Run the script...
  python daily-breakdowns.py
#+end_src

* Design/Trade-Off Notes and Decisions

1. This script is the main one regarding the chart generation part of the
   project.
   1. The original =../data/lm1-exhibiton-all.csv= and
      =../data/lm2-exhibiton-all.csv= files are too large to work with on average
      hardware. So, I had to split it them up (using ~./separator.sh~) into
      smaller files. If you are use the CSV files (found in =../data=) with
      Microsoft Excel, I recommend the ones listed in the [[https://git.abbether.net/return-to-ritherdon/ritherdon-charts/src/branch/master/docs/daily-breakdowns.org#headline-3][Files Used]] section
      below.
2. Each day (for each Light Meter) has its data processed and a chart is made
   from that data. This is cause for the bulk of the files produced in the
   =../output= directory.
3. The reason for line charts is because Nicola would like to use the lines
   formed in these charts for an artwork.
   1. Ideally, she would prefer to use the files containing all the data, for a
      given Light Meter, but they are too cumbersome. Splitting the original CSV
      files into smaller (daily) ones was the preferred compromise.
4. The charts produced for Light Meter 1 use /blue/ lines and /orange/ for Light
   Meter 2. I standardised on these two colours to help quickly identify the
   Light Meter depicted in a given chart. There is no specific reasons for the
   colours chosen.
5. The exhibition ran until the 1^st August 2021, which a Sunday. Ritherdon was
   not open that day so no readings were recorded. I have omitted the files for
   that date because of this.
   1. It was easier to stop the =separator.sh= script from splitting files up at
      the end of July (2021) than extending it for an extra day in August
      (2021). This is especially true when you realise there is no usable data
      for 1^st August.
6. The data illustrated in all the charts (produced with this script) consist of
   the times each Light Meter was on for a given day. Both Light Meters were
   sometimes not turned on during working hours (in Ritherdon); This is the main
   cause for the inconsistencies across the x-axis timestamps in the charts
   produced by this script. Basically, the data is messy and the results show
   some of the mess.
7. The staff at Ritherdon turned the Light Meters on and kept them on until they
   finished their shift. Technically, the Light Meters could have been turned on
   and off multiple times a day but that behaviour never happened. The code in
   this script makes the assumption each day was one continuous series of
   readings because of this. The code in this script would need to change if
   that were not the case.

* Files Used

#+begin_src shell :results code
  ls ../data/light-meter-*/*.csv
#+end_src

#+RESULTS:
#+begin_src shell
../data/light-meter-1/2021-06-13.csv
../data/light-meter-1/2021-06-14.csv
../data/light-meter-1/2021-06-15.csv
../data/light-meter-1/2021-06-16.csv
../data/light-meter-1/2021-06-17.csv
../data/light-meter-1/2021-06-18.csv
../data/light-meter-1/2021-06-19.csv
../data/light-meter-1/2021-06-20.csv
../data/light-meter-1/2021-06-21.csv
../data/light-meter-1/2021-06-22.csv
../data/light-meter-1/2021-06-23.csv
../data/light-meter-1/2021-06-24.csv
../data/light-meter-1/2021-06-25.csv
../data/light-meter-1/2021-06-26.csv
../data/light-meter-1/2021-06-27.csv
../data/light-meter-1/2021-06-28.csv
../data/light-meter-1/2021-06-29.csv
../data/light-meter-1/2021-06-30.csv
../data/light-meter-1/2021-07-01.csv
../data/light-meter-1/2021-07-02.csv
../data/light-meter-1/2021-07-03.csv
../data/light-meter-1/2021-07-04.csv
../data/light-meter-1/2021-07-05.csv
../data/light-meter-1/2021-07-06.csv
../data/light-meter-1/2021-07-07.csv
../data/light-meter-1/2021-07-08.csv
../data/light-meter-1/2021-07-09.csv
../data/light-meter-1/2021-07-10.csv
../data/light-meter-1/2021-07-11.csv
../data/light-meter-1/2021-07-12.csv
../data/light-meter-1/2021-07-13.csv
../data/light-meter-1/2021-07-14.csv
../data/light-meter-1/2021-07-15.csv
../data/light-meter-1/2021-07-16.csv
../data/light-meter-1/2021-07-17.csv
../data/light-meter-1/2021-07-18.csv
../data/light-meter-1/2021-07-19.csv
../data/light-meter-1/2021-07-20.csv
../data/light-meter-1/2021-07-21.csv
../data/light-meter-1/2021-07-22.csv
../data/light-meter-1/2021-07-23.csv
../data/light-meter-1/2021-07-24.csv
../data/light-meter-1/2021-07-25.csv
../data/light-meter-1/2021-07-26.csv
../data/light-meter-1/2021-07-27.csv
../data/light-meter-1/2021-07-28.csv
../data/light-meter-1/2021-07-29.csv
../data/light-meter-1/2021-07-30.csv
../data/light-meter-2/2021-06-13.csv
../data/light-meter-2/2021-06-14.csv
../data/light-meter-2/2021-06-15.csv
../data/light-meter-2/2021-06-16.csv
../data/light-meter-2/2021-06-17.csv
../data/light-meter-2/2021-06-18.csv
../data/light-meter-2/2021-06-19.csv
../data/light-meter-2/2021-06-20.csv
../data/light-meter-2/2021-06-21.csv
../data/light-meter-2/2021-06-22.csv
../data/light-meter-2/2021-06-23.csv
../data/light-meter-2/2021-06-24.csv
../data/light-meter-2/2021-06-25.csv
../data/light-meter-2/2021-06-26.csv
../data/light-meter-2/2021-06-27.csv
../data/light-meter-2/2021-06-28.csv
../data/light-meter-2/2021-06-29.csv
../data/light-meter-2/2021-06-30.csv
../data/light-meter-2/2021-07-01.csv
../data/light-meter-2/2021-07-02.csv
../data/light-meter-2/2021-07-03.csv
../data/light-meter-2/2021-07-04.csv
../data/light-meter-2/2021-07-05.csv
../data/light-meter-2/2021-07-06.csv
../data/light-meter-2/2021-07-07.csv
../data/light-meter-2/2021-07-08.csv
../data/light-meter-2/2021-07-09.csv
../data/light-meter-2/2021-07-10.csv
../data/light-meter-2/2021-07-11.csv
../data/light-meter-2/2021-07-12.csv
../data/light-meter-2/2021-07-13.csv
../data/light-meter-2/2021-07-14.csv
../data/light-meter-2/2021-07-15.csv
../data/light-meter-2/2021-07-16.csv
../data/light-meter-2/2021-07-17.csv
../data/light-meter-2/2021-07-18.csv
../data/light-meter-2/2021-07-19.csv
../data/light-meter-2/2021-07-20.csv
../data/light-meter-2/2021-07-21.csv
../data/light-meter-2/2021-07-22.csv
../data/light-meter-2/2021-07-23.csv
../data/light-meter-2/2021-07-24.csv
../data/light-meter-2/2021-07-25.csv
../data/light-meter-2/2021-07-26.csv
../data/light-meter-2/2021-07-27.csv
../data/light-meter-2/2021-07-28.csv
../data/light-meter-2/2021-07-29.csv
../data/light-meter-2/2021-07-30.csv
#+end_src

* Files Produced

#+begin_src shell :results code
  ls ../output/lm*-2021-*-*.html
#+end_src

#+RESULTS:
#+begin_src shell
../output/lm1-2021-06-13.html
../output/lm1-2021-06-14.html
../output/lm1-2021-06-15.html
../output/lm1-2021-06-16.html
../output/lm1-2021-06-17.html
../output/lm1-2021-06-18.html
../output/lm1-2021-06-19.html
../output/lm1-2021-06-20.html
../output/lm1-2021-06-21.html
../output/lm1-2021-06-22.html
../output/lm1-2021-06-23.html
../output/lm1-2021-06-24.html
../output/lm1-2021-06-25.html
../output/lm1-2021-06-26.html
../output/lm1-2021-06-27.html
../output/lm1-2021-06-28.html
../output/lm1-2021-06-29.html
../output/lm1-2021-06-30.html
../output/lm1-2021-07-01.html
../output/lm1-2021-07-02.html
../output/lm1-2021-07-03.html
../output/lm1-2021-07-04.html
../output/lm1-2021-07-05.html
../output/lm1-2021-07-06.html
../output/lm1-2021-07-07.html
../output/lm1-2021-07-08.html
../output/lm1-2021-07-09.html
../output/lm1-2021-07-10.html
../output/lm1-2021-07-11.html
../output/lm1-2021-07-12.html
../output/lm1-2021-07-13.html
../output/lm1-2021-07-14.html
../output/lm1-2021-07-15.html
../output/lm1-2021-07-16.html
../output/lm1-2021-07-17.html
../output/lm1-2021-07-18.html
../output/lm1-2021-07-19.html
../output/lm1-2021-07-20.html
../output/lm1-2021-07-21.html
../output/lm1-2021-07-22.html
../output/lm1-2021-07-23.html
../output/lm1-2021-07-24.html
../output/lm1-2021-07-25.html
../output/lm1-2021-07-26.html
../output/lm1-2021-07-27.html
../output/lm1-2021-07-28.html
../output/lm1-2021-07-29.html
../output/lm1-2021-07-30.html
../output/lm2-2021-06-13.html
../output/lm2-2021-06-14.html
../output/lm2-2021-06-15.html
../output/lm2-2021-06-16.html
../output/lm2-2021-06-17.html
../output/lm2-2021-06-18.html
../output/lm2-2021-06-19.html
../output/lm2-2021-06-20.html
../output/lm2-2021-06-21.html
../output/lm2-2021-06-22.html
../output/lm2-2021-06-23.html
../output/lm2-2021-06-24.html
../output/lm2-2021-06-25.html
../output/lm2-2021-06-26.html
../output/lm2-2021-06-27.html
../output/lm2-2021-06-28.html
../output/lm2-2021-06-29.html
../output/lm2-2021-06-30.html
../output/lm2-2021-07-01.html
../output/lm2-2021-07-02.html
../output/lm2-2021-07-03.html
../output/lm2-2021-07-04.html
../output/lm2-2021-07-05.html
../output/lm2-2021-07-06.html
../output/lm2-2021-07-07.html
../output/lm2-2021-07-08.html
../output/lm2-2021-07-09.html
../output/lm2-2021-07-10.html
../output/lm2-2021-07-11.html
../output/lm2-2021-07-12.html
../output/lm2-2021-07-13.html
../output/lm2-2021-07-14.html
../output/lm2-2021-07-15.html
../output/lm2-2021-07-16.html
../output/lm2-2021-07-17.html
../output/lm2-2021-07-18.html
../output/lm2-2021-07-19.html
../output/lm2-2021-07-20.html
../output/lm2-2021-07-21.html
../output/lm2-2021-07-22.html
../output/lm2-2021-07-23.html
../output/lm2-2021-07-24.html
../output/lm2-2021-07-25.html
../output/lm2-2021-07-26.html
../output/lm2-2021-07-27.html
../output/lm2-2021-07-28.html
../output/lm2-2021-07-29.html
../output/lm2-2021-07-30.html
#+end_src

* What the Script Does

This script produces a series of charts illustrating the readings taken for each
day the exhibition was running (excluding the 1^st August 2021). It does this by
reading all the files listed above in the [[https://git.abbether.net/return-to-ritherdon/ritherdon-charts/src/branch/master/docs/daily-breakdowns.org#headline-3][Files Used]] section. It processes the
data for both Light Meters, also. Conceptually, there is less faffing in this
script than in some of the others. It takes the values in the ~time~ column (in
the CSV files) for the x-axis and the values in the ~reading~ column for the
y-axis. Below is an example of the what the data looks like in the CSV
files. The example is from =../data/light-meter-1/2021-06-14.csv=.

| time                       | reading |
|----------------------------+---------|
| 2021-06-14 06:45:54.000000 |    -111 |
| 2021-06-14 06:45:58.000000 |       0 |
| 2021-06-14 06:46:01.000000 |       2 |
| 2021-06-14 06:46:03.000000 |       3 |
| 2021-06-14 06:46:06.000000 |       3 |
| 2021-06-14 06:46:08.000000 |       4 |
| 2021-06-14 06:46:11.000000 |       4 |
| 2021-06-14 06:46:13.000000 |       4 |
| 2021-06-14 06:46:15.000000 |       5 |
| MORE READINGS HERE         |     ... |
| 2021-06-14 15:59:59.000000 |      17 |
| 2021-06-14 16:00:00.000000 |      17 |
| 2021-06-14 16:00:01.000000 |      17 |

When the script has finishing processing a file, you should get a chart with a
single continuous line on it containing light readings for the entire day
(excluding non-work hours).

* Examples/Screenshots

- Blue used for Ligth Meter 1 charts
- Orange used for Light Meter 2 charts

[[file:../assets/lm1-readings-for-2021-06-28.png]]

[[file:../assets/lm2-readings-for-2021-07-22.png]]