This script takes the 'same' day from each month (E.G. the 16th June and July)
and overlays each others charts on top of each other.
If there is no data for both days (E.G. data for the 15th of June but not for
the 15th July), no chart is produced.
Because each chart has inconsistent timestamps (between the two days), each
reading is given a 'Reading Id.' so the time element of each chart is lost: THEY
ARE NOT ONE-TO-ONE MAPPING OF EACH DAY. The charts produced with this script
produce charts depicting the amount of change as a set of sequential events and
that's it. They show a general pattern to the day. For example, a chart will
show an increase in light from point A to Point B for Light Meter 1. It will not
specify how long it took for the change to occur.
This script places all the daily reading for a given Light Meter and overlays
them on top of each other on the chart. Because each file (for each day)
have different combinations of date/timestamps, the charts has to switch to
using plain integers for the x-axis. This means you can't glean much data from a
time/date spective but the charts for both Light Meters do help relay the
amount of activity which happened in them.
I didn't mean to create this script.
It takes the individual daily readings for a Light Meter and places each 'daily
readings' next to each other, on the same graph. This is essentially stitching
the full log of reading back together but with gaps between the individual
charts. The gaps are formed, I think -- I haven't checked -- because the
'missing data' (when the factory was closed or no welding was happening) is
being treated as such. I'm assuming it's because I've passed each 'daily line'
as separate lines so Bokeh has no reason to join them together.
This script makes a HTML chart for both Light Meters. And, the charts are slow
and not very nice to deal with. I've only kept the script and the charts it
produces because the charts look interesting as images and I want to see what
Nic thinks. She might find them useful from an artistic perspective.
The script now processes all the daily files for both Light Meters. Each day,
for each Light Meter, has their own chart (HTML) made as the output of the
script.
The script can process on file and create plot (HTML file) but the aim is to
have the script cycle through all the daily files (for Light Meter 1) and
produce charts of each one.
This script takes all the individual files in /data/light-meter-1-hourly-totals
and creates a line chart from the data. Each line tracks the number of readings
taken in a given hour and each line represents one day during the exhibitions
run time (13th June 2021 to 1st August 2021). The lines are layered on top of
each other in a single grid.
This script produces a line chart, using Bokeh, displaying the daily totals for
both Light Meters. The data is process from /data and the chart is written to
the /output directory -- as a HTML file.
I added checks to stop this script from processing data which was not recorded
during the exhibition's opening times.
Note: The exhibition officially closes on 1st August 2021 but that was a Sunday
and no one was working in Ritherdon that day. This is why the looping range has
been reduced from 6-8 to 6-7.
The exhibition officially ended 1st August 2021 but no one was working in
Ritherdon on that day (Sunday). So, there was not light readings taken. This
script originally was extracting data all the way up to the 31st August
2021. This meant the old version of this script was extracting excess data and
producing files with irrelevant data. The changes here remove the excess data
extraction and processing.
This script processes the files in the light-meter-1/ and light-meter-2/
directories in the data/ directory. It tallies up the total readings taken for a
given day and the individual hours in a given day.
Note: You must run this code after you've ran the separator.sh script.
The script was processing lm1-exhibiton-all.csv twice because I forgot to
replace the hardcoded value ('lm1') in the path used by rg. The code both
processes lm1-exhibition-all.csv and lm2-exhibition-all.csv files.
For some reason, I have a habit of spelling separator as 'seperator' and it
takes my a while to notice the mistake.
Clean-up involves adding comments and removing unused code.
You can now pass either a '1' or '2' to select which Light Meter data you want
to process -- all in the same script. This allows you to process both sets of
data at the same time.
Each Meters' data is place in their own directory in data/ so you can work on
each one seperately without mixing them up.
I've been using this outside of the repo. to extract the data from the
database (DB not in repository). I'm adding it now because it's getting to point
which is no longer 'throw-away' code (it's got comments now and everything). So,
I'm adding it here so I don't lose it.
This CSV file contains all the reading taken by Light Meter 2, during the
exhibiton. This is approx. 96MB so it makes Git run a bit slow -- to be fair,
Kate, Libre Office and anything I've opened this file with is just slow when
working with it.
This shell script takes the data from data/lm1-exhibition-all.csv and breaks it
down into readings taken on a daily scale and then an hourly scale.
At the moment, it produces directories and files for times/days which have no
readings but I don't know if that's relevant or not. By having no data in
certain files, it does confirm 'No Data Recorded' if/when people look to review
the data and wonder why they can't find any for a particular day.
This is all the readings taken during the No Gaps in the Line exhibition for
Light Meter 1. This is a large CSV file (approx. 60MB) and is a pain to work
with on average spec'd computers. The aim is to break this file down into
various files focusing on daily hourly and monthly reading. This should make
working with the data in this file easier.
NOTE: There is a second Light Meter but the data has not been added to the
repository yet. The plan is to get Light Meter 1's data working then focus on
the Light Meter 2 -- hopefully it'll be as easy as just changing the default
file in the code.
Added directories to the place the data will be written to. The files created
will be numerous and large so this helps keep the repo. and commit history tidy.
