return-to-ritherdon
/
ritherdon-charts
1
0
Fork 0
Code Releases 1 Activity
Data processing and plotting for 'Personal Flash' artworks. https://www.nicolaellisandritherdon.com
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
63 Commits
1 Branch
1 Tag
13 MiB
 
 
Branch: master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
ritherdon-charts/docs/daily-breakdowns.org

11 KiB
Raw Permalink Blame History

Overview of daily-breakdowns.py Script

I have assumed you have ran ./separator.sh and ./totalilator.sh, created a virtual-environment (venv) (python3 -m venv venv) and installed the dependencies (via pip -r install requirements.txt). See the project's README for more information on setting the repository up.

  cd <PATH TO RITHERDON-CHARTS>
  # Activate the virtual-environment if you haven't...
  source venv/bin/activate

  # Run the script...
  python daily-breakdowns.py

Design/Trade-Off Notes and Decisions

  1. This script is the main one regarding the chart generation part of the project.

    1. The original ../data/lm1-exhibiton-all.csv and ../data/lm2-exhibiton-all.csv files are too large to work with on average hardware. So, I had to split it them up (using ./separator.sh) into smaller files. If you are use the CSV files (found in ../data) with Microsoft Excel, I recommend the ones listed in the Files Used section below.
  2. Each day (for each Light Meter) has its data processed and a chart is made from that data. This is cause for the bulk of the files produced in the ../output directory.

  3. The reason for line charts is because Nicola would like to use the lines formed in these charts for an artwork.

    1. Ideally, she would prefer to use the files containing all the data, for a given Light Meter, but they are too cumbersome. Splitting the original CSV files into smaller (daily) ones was the preferred compromise.
  4. The charts produced for Light Meter 1 use blue lines and orange for Light Meter 2. I standardised on these two colours to help quickly identify the Light Meter depicted in a given chart. There is no specific reasons for the colours chosen.

  5. The exhibition ran until the 1^st August 2021, which a Sunday. Ritherdon was not open that day so no readings were recorded. I have omitted the files for that date because of this.

    1. It was easier to stop the separator.sh script from splitting files up at the end of July (2021) than extending it for an extra day in August (2021). This is especially true when you realise there is no usable data for 1^st August.
  6. The data illustrated in all the charts (produced with this script) consist of the times each Light Meter was on for a given day. Both Light Meters were sometimes not turned on during working hours (in Ritherdon); This is the main cause for the inconsistencies across the x-axis timestamps in the charts produced by this script. Basically, the data is messy and the results show some of the mess.
  7. The staff at Ritherdon turned the Light Meters on and kept them on until they finished their shift. Technically, the Light Meters could have been turned on and off multiple times a day but that behaviour never happened. The code in this script makes the assumption each day was one continuous series of readings because of this. The code in this script would need to change if that were not the case.

Files Used 

  ls ../data/light-meter-*/*.csv
../data/light-meter-1/2021-06-13.csv
../data/light-meter-1/2021-06-14.csv
../data/light-meter-1/2021-06-15.csv
../data/light-meter-1/2021-06-16.csv
../data/light-meter-1/2021-06-17.csv
../data/light-meter-1/2021-06-18.csv
../data/light-meter-1/2021-06-19.csv
../data/light-meter-1/2021-06-20.csv
../data/light-meter-1/2021-06-21.csv
../data/light-meter-1/2021-06-22.csv
../data/light-meter-1/2021-06-23.csv
../data/light-meter-1/2021-06-24.csv
../data/light-meter-1/2021-06-25.csv
../data/light-meter-1/2021-06-26.csv
../data/light-meter-1/2021-06-27.csv
../data/light-meter-1/2021-06-28.csv
../data/light-meter-1/2021-06-29.csv
../data/light-meter-1/2021-06-30.csv
../data/light-meter-1/2021-07-01.csv
../data/light-meter-1/2021-07-02.csv
../data/light-meter-1/2021-07-03.csv
../data/light-meter-1/2021-07-04.csv
../data/light-meter-1/2021-07-05.csv
../data/light-meter-1/2021-07-06.csv
../data/light-meter-1/2021-07-07.csv
../data/light-meter-1/2021-07-08.csv
../data/light-meter-1/2021-07-09.csv
../data/light-meter-1/2021-07-10.csv
../data/light-meter-1/2021-07-11.csv
../data/light-meter-1/2021-07-12.csv
../data/light-meter-1/2021-07-13.csv
../data/light-meter-1/2021-07-14.csv
../data/light-meter-1/2021-07-15.csv
../data/light-meter-1/2021-07-16.csv
../data/light-meter-1/2021-07-17.csv
../data/light-meter-1/2021-07-18.csv
../data/light-meter-1/2021-07-19.csv
../data/light-meter-1/2021-07-20.csv
../data/light-meter-1/2021-07-21.csv
../data/light-meter-1/2021-07-22.csv
../data/light-meter-1/2021-07-23.csv
../data/light-meter-1/2021-07-24.csv
../data/light-meter-1/2021-07-25.csv
../data/light-meter-1/2021-07-26.csv
../data/light-meter-1/2021-07-27.csv
../data/light-meter-1/2021-07-28.csv
../data/light-meter-1/2021-07-29.csv
../data/light-meter-1/2021-07-30.csv
../data/light-meter-2/2021-06-13.csv
../data/light-meter-2/2021-06-14.csv
../data/light-meter-2/2021-06-15.csv
../data/light-meter-2/2021-06-16.csv
../data/light-meter-2/2021-06-17.csv
../data/light-meter-2/2021-06-18.csv
../data/light-meter-2/2021-06-19.csv
../data/light-meter-2/2021-06-20.csv
../data/light-meter-2/2021-06-21.csv
../data/light-meter-2/2021-06-22.csv
../data/light-meter-2/2021-06-23.csv
../data/light-meter-2/2021-06-24.csv
../data/light-meter-2/2021-06-25.csv
../data/light-meter-2/2021-06-26.csv
../data/light-meter-2/2021-06-27.csv
../data/light-meter-2/2021-06-28.csv
../data/light-meter-2/2021-06-29.csv
../data/light-meter-2/2021-06-30.csv
../data/light-meter-2/2021-07-01.csv
../data/light-meter-2/2021-07-02.csv
../data/light-meter-2/2021-07-03.csv
../data/light-meter-2/2021-07-04.csv
../data/light-meter-2/2021-07-05.csv
../data/light-meter-2/2021-07-06.csv
../data/light-meter-2/2021-07-07.csv
../data/light-meter-2/2021-07-08.csv
../data/light-meter-2/2021-07-09.csv
../data/light-meter-2/2021-07-10.csv
../data/light-meter-2/2021-07-11.csv
../data/light-meter-2/2021-07-12.csv
../data/light-meter-2/2021-07-13.csv
../data/light-meter-2/2021-07-14.csv
../data/light-meter-2/2021-07-15.csv
../data/light-meter-2/2021-07-16.csv
../data/light-meter-2/2021-07-17.csv
../data/light-meter-2/2021-07-18.csv
../data/light-meter-2/2021-07-19.csv
../data/light-meter-2/2021-07-20.csv
../data/light-meter-2/2021-07-21.csv
../data/light-meter-2/2021-07-22.csv
../data/light-meter-2/2021-07-23.csv
../data/light-meter-2/2021-07-24.csv
../data/light-meter-2/2021-07-25.csv
../data/light-meter-2/2021-07-26.csv
../data/light-meter-2/2021-07-27.csv
../data/light-meter-2/2021-07-28.csv
../data/light-meter-2/2021-07-29.csv
../data/light-meter-2/2021-07-30.csv

Files Produced 

  ls ../output/lm*-2021-*-*.html
../output/lm1-2021-06-13.html
../output/lm1-2021-06-14.html
../output/lm1-2021-06-15.html
../output/lm1-2021-06-16.html
../output/lm1-2021-06-17.html
../output/lm1-2021-06-18.html
../output/lm1-2021-06-19.html
../output/lm1-2021-06-20.html
../output/lm1-2021-06-21.html
../output/lm1-2021-06-22.html
../output/lm1-2021-06-23.html
../output/lm1-2021-06-24.html
../output/lm1-2021-06-25.html
../output/lm1-2021-06-26.html
../output/lm1-2021-06-27.html
../output/lm1-2021-06-28.html
../output/lm1-2021-06-29.html
../output/lm1-2021-06-30.html
../output/lm1-2021-07-01.html
../output/lm1-2021-07-02.html
../output/lm1-2021-07-03.html
../output/lm1-2021-07-04.html
../output/lm1-2021-07-05.html
../output/lm1-2021-07-06.html
../output/lm1-2021-07-07.html
../output/lm1-2021-07-08.html
../output/lm1-2021-07-09.html
../output/lm1-2021-07-10.html
../output/lm1-2021-07-11.html
../output/lm1-2021-07-12.html
../output/lm1-2021-07-13.html
../output/lm1-2021-07-14.html
../output/lm1-2021-07-15.html
../output/lm1-2021-07-16.html
../output/lm1-2021-07-17.html
../output/lm1-2021-07-18.html
../output/lm1-2021-07-19.html
../output/lm1-2021-07-20.html
../output/lm1-2021-07-21.html
../output/lm1-2021-07-22.html
../output/lm1-2021-07-23.html
../output/lm1-2021-07-24.html
../output/lm1-2021-07-25.html
../output/lm1-2021-07-26.html
../output/lm1-2021-07-27.html
../output/lm1-2021-07-28.html
../output/lm1-2021-07-29.html
../output/lm1-2021-07-30.html
../output/lm2-2021-06-13.html
../output/lm2-2021-06-14.html
../output/lm2-2021-06-15.html
../output/lm2-2021-06-16.html
../output/lm2-2021-06-17.html
../output/lm2-2021-06-18.html
../output/lm2-2021-06-19.html
../output/lm2-2021-06-20.html
../output/lm2-2021-06-21.html
../output/lm2-2021-06-22.html
../output/lm2-2021-06-23.html
../output/lm2-2021-06-24.html
../output/lm2-2021-06-25.html
../output/lm2-2021-06-26.html
../output/lm2-2021-06-27.html
../output/lm2-2021-06-28.html
../output/lm2-2021-06-29.html
../output/lm2-2021-06-30.html
../output/lm2-2021-07-01.html
../output/lm2-2021-07-02.html
../output/lm2-2021-07-03.html
../output/lm2-2021-07-04.html
../output/lm2-2021-07-05.html
../output/lm2-2021-07-06.html
../output/lm2-2021-07-07.html
../output/lm2-2021-07-08.html
../output/lm2-2021-07-09.html
../output/lm2-2021-07-10.html
../output/lm2-2021-07-11.html
../output/lm2-2021-07-12.html
../output/lm2-2021-07-13.html
../output/lm2-2021-07-14.html
../output/lm2-2021-07-15.html
../output/lm2-2021-07-16.html
../output/lm2-2021-07-17.html
../output/lm2-2021-07-18.html
../output/lm2-2021-07-19.html
../output/lm2-2021-07-20.html
../output/lm2-2021-07-21.html
../output/lm2-2021-07-22.html
../output/lm2-2021-07-23.html
../output/lm2-2021-07-24.html
../output/lm2-2021-07-25.html
../output/lm2-2021-07-26.html
../output/lm2-2021-07-27.html
../output/lm2-2021-07-28.html
../output/lm2-2021-07-29.html
../output/lm2-2021-07-30.html

What the Script Does

This script produces a series of charts illustrating the readings taken for each day the exhibition was running (excluding the 1^st August 2021). It does this by reading all the files listed above in the Files Used section. It processes the data for both Light Meters, also. Conceptually, there is less faffing in this script than in some of the others. It takes the values in the time column (in the CSV files) for the x-axis and the values in the reading column for the y-axis. Below is an example of the what the data looks like in the CSV files. The example is from ../data/light-meter-1/2021-06-14.csv.

time reading
2021-06-14 06:45:54.000000 -111
2021-06-14 06:45:58.000000 0
2021-06-14 06:46:01.000000 2
2021-06-14 06:46:03.000000 3
2021-06-14 06:46:06.000000 3
2021-06-14 06:46:08.000000 4
2021-06-14 06:46:11.000000 4
2021-06-14 06:46:13.000000 4
2021-06-14 06:46:15.000000 5
MORE READINGS HERE …
2021-06-14 15:59:59.000000 17
2021-06-14 16:00:00.000000 17
2021-06-14 16:00:01.000000 17

When the script has finishing processing a file, you should get a chart with a single continuous line on it containing light readings for the entire day (excluding non-work hours).

Examples/Screenshots

  • Blue used for Ligth Meter 1 charts
  • Orange used for Light Meter 2 charts

/return-to-ritherdon/ritherdon-charts/src/branch/master/assets/lm1-readings-for-2021-06-28.png

/return-to-ritherdon/ritherdon-charts/src/branch/master/assets/lm2-readings-for-2021-07-22.png