The script now processes all the daily files for both Light Meters. Each day,
for each Light Meter, has their own chart (HTML) made as the output of the
script.
The script can process on file and create plot (HTML file) but the aim is to
have the script cycle through all the daily files (for Light Meter 1) and
produce charts of each one.
This script takes all the individual files in /data/light-meter-1-hourly-totals
and creates a line chart from the data. Each line tracks the number of readings
taken in a given hour and each line represents one day during the exhibitions
run time (13th June 2021 to 1st August 2021). The lines are layered on top of
each other in a single grid.
This script produces a line chart, using Bokeh, displaying the daily totals for
both Light Meters. The data is process from /data and the chart is written to
the /output directory -- as a HTML file.
I added checks to stop this script from processing data which was not recorded
during the exhibition's opening times.
Note: The exhibition officially closes on 1st August 2021 but that was a Sunday
and no one was working in Ritherdon that day. This is why the looping range has
been reduced from 6-8 to 6-7.
The exhibition officially ended 1st August 2021 but no one was working in
Ritherdon on that day (Sunday). So, there was not light readings taken. This
script originally was extracting data all the way up to the 31st August
2021. This meant the old version of this script was extracting excess data and
producing files with irrelevant data. The changes here remove the excess data
extraction and processing.
This script processes the files in the light-meter-1/ and light-meter-2/
directories in the data/ directory. It tallies up the total readings taken for a
given day and the individual hours in a given day.
Note: You must run this code after you've ran the separator.sh script.
The script was processing lm1-exhibiton-all.csv twice because I forgot to
replace the hardcoded value ('lm1') in the path used by rg. The code both
processes lm1-exhibition-all.csv and lm2-exhibition-all.csv files.
For some reason, I have a habit of spelling separator as 'seperator' and it
takes my a while to notice the mistake.
Clean-up involves adding comments and removing unused code.
You can now pass either a '1' or '2' to select which Light Meter data you want
to process -- all in the same script. This allows you to process both sets of
data at the same time.
Each Meters' data is place in their own directory in data/ so you can work on
each one seperately without mixing them up.
I've been using this outside of the repo. to extract the data from the
database (DB not in repository). I'm adding it now because it's getting to point
which is no longer 'throw-away' code (it's got comments now and everything). So,
I'm adding it here so I don't lose it.
This CSV file contains all the reading taken by Light Meter 2, during the
exhibiton. This is approx. 96MB so it makes Git run a bit slow -- to be fair,
Kate, Libre Office and anything I've opened this file with is just slow when
working with it.
This shell script takes the data from data/lm1-exhibition-all.csv and breaks it
down into readings taken on a daily scale and then an hourly scale.
At the moment, it produces directories and files for times/days which have no
readings but I don't know if that's relevant or not. By having no data in
certain files, it does confirm 'No Data Recorded' if/when people look to review
the data and wonder why they can't find any for a particular day.
This is all the readings taken during the No Gaps in the Line exhibition for
Light Meter 1. This is a large CSV file (approx. 60MB) and is a pain to work
with on average spec'd computers. The aim is to break this file down into
various files focusing on daily hourly and monthly reading. This should make
working with the data in this file easier.
NOTE: There is a second Light Meter but the data has not been added to the
repository yet. The plan is to get Light Meter 1's data working then focus on
the Light Meter 2 -- hopefully it'll be as easy as just changing the default
file in the code.
Added directories to the place the data will be written to. The files created
will be numerous and large so this helps keep the repo. and commit history tidy.
Craig Oates