Browse Source

update README.org file.

master
Craig Oates 1 year ago
parent
commit
df2bac4ab4
  1. 113
      README.org

113
README.org

@ -18,13 +18,14 @@ Gaps in the Line]] exhibition by [[http://www.nicolaellis.com][Nicola Ellis]], h
Manchester, U.K.
This project ties into a larger collection of software projects related to the
/Personal Flash in Real-Time/ artworks. Which, in turn, is a small piece of the
much larger [[https://www.nicolaellisandritherdon.com/][Return to Ritherdon]] project (devised and completed by Nicola
Ellis). For more information on the software-side of the /Personal Flash in
Real-Time/ artworks, please use the links below:
/Personal Flash in Real-Time/ artworks. And, those artworks are a small piece of
the much larger [[https://www.nicolaellisandritherdon.com/][Return to Ritherdon]] project (devised and completed by Nicola
Ellis). For more information on the artworks and where they sit in the larger
project, please use the links below:
- [[https://git.abbether.net/return-to-ritherdon/rtr-docs][rtr-docs]] (The documentation repository for all the /Person Flash in Real-Time/
software projects)
software projects, with an overview of how they tie into the Return to
Ritherdon project)
- [[https://git.abbether.net/return-to-ritherdon][Return to Ritherdon Org. Page]] (The 'home page' for the /Return to Ritherdon/
project on this site, containing a list of all the publicly available
repositories)
@ -35,16 +36,16 @@ manufacturing electrical enclosures and other related products. So, if you have
spent any time in the U.K. and seen one of those green electrical boxes lurking
on a street corner, there is a good chance these folks made it.
*NOTE: This project does not contain documentation in the [[https://git.abbether.net/return-to-ritherdon/rtr-docs][rtr-docs]] repository.* It
is a self-contained mini-project which is not directly related to the /Personal
Flash in Real-Time/ artworks.
*NOTE: This project does not contain documentation in the [[https://git.abbether.net/return-to-ritherdon/rtr-docs][rtr-docs]] repository.*
This is a self-contained mini-project which is not /directly related/ to the
/Personal Flash in Real-Time/ artworks.
*** Examples/Screenshots
At the time of writing, the scripts in this repository produce over one-hundred
charts/files. So, here are a selection of the types of charts produced after
processing the data in =data/lm1-exhibiton-all.csv= and
=/data/lm2-exhibition-all.csv=.
processing the data in the =data/lm1-exhibiton-all.csv= and
=/data/lm2-exhibition-all.csv= files.
[[file:assets/daily-totals.png]]
@ -59,14 +60,14 @@ processing the data in =data/lm1-exhibiton-all.csv= and
** Overview of the /Personal Flash in Real-Time/ Artworks
/Personal Flash in Real-Time/ consists of two artworks, named /Personal Flash in
Real-Time (Andy)/ and /Personal Flash in Real-Time (Tony)/. Each one measured the
light in the welding booths in the Ritherdon Factory and forwarded those
Real-Time (Andy)/ and /Personal Flash in Real-Time (Tony)/. Each one measured
the light in the welding booths in the Ritherdon Factory and forwarded those
readings on to a server running in Amazon's 'cloud' -- see [[https://aws.amazon.com/][Amazon Web Services]]
(/AWS/) for more information. While this was happening, two sets of lights
residing in Castlefield Gallery would turn on and off whenever the system
(AWS) for more information. While this was happening, two sets of lights,
residing in Castlefield Gallery, would turn on and off whenever the system
detected someone welding in Ritherdon. The would happen because the Relays,
controlling the lights would receive the latest Light Meter readings taken in
Ritherdon via the server (/AWS/).
Ritherdon via the server (AWS).
The (AWS) server stored every reading taken in a SQLite database and this
project pokes and prods at the data -- to plot charts/graphs.
@ -79,10 +80,10 @@ project pokes and prods at the data -- to plot charts/graphs.
2. The =separator.sh= and =totalilator.sh= scripts split the .csv files, mentioned
above, into smaller files in an attempt to make them (.csv files) easier to
work with on average hardware.
1. On top of that,I have only committed the .csv files mentioned in point 1
1. On top of that,I have only committed the CSV files mentioned in point 1
to the repository as a means to reduce the clutter in the repositories Git
commit history.
2. You will need to split the .csv files up yourself after you have cloned
2. You will need to split the CSV files up yourself after you have cloned
the repository, using the scripts mentioned in point 2.
3. *The database containing the actual data is not included with this repository.*
1. The database used for the /No Gaps in the Line/ exhibition is approximately
@ -92,28 +93,28 @@ project pokes and prods at the data -- to plot charts/graphs.
(13^th June, 2021) and 2021-08-01 (1^st August, 2021) for both Light Meters
(the length of the exhibition): There is (not much) more (test) data in the
database but the data selected/exported seemed the most appropriate decision.
5. I chose to work with .csv files out of convenience more than anything else --
5. I chose to work with CSV files out of convenience more than anything else --
it is the easiest format to export the data to from the SQlite database.
6. I used [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)][Bash]], [[https://en.wikipedia.org/wiki/AWK][Awk]] and [[https://github.com/BurntSushi/ripgrep][Ripgrep]] (/rg/), also, out of convenience, they were
6. I used [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)][Bash]], [[https://en.wikipedia.org/wiki/AWK][Awk]] and [[https://github.com/BurntSushi/ripgrep][Ripgrep]] (rg), also, out of convenience, they were
already on my computer.
7. I used [[https://bokeh.org/][Bokeh]] because I have already used it and it is the only thing I know
which can create interactive charts as individual HTML files, which I can
share with anyone not comfortable with computers.
8. I used [[https://www.python.org/][Python]] because of Bokeh.
9. Overall, Nicola wants to work with the charts this data produces so
any decisions made should be in service to that end.
9. Overall, Nicola wants to work with the charts produced by this data so any
decisions made should be in service to that end.
10. I have taken a hard-coded approach to filenames with the code because the
code is not the main objective here, the charts are; *Long-term flexibility
and maintenance is not a concern here.*
code is not the main objective here, the charts are; In other words,
*long-term flexibility and maintenance is not a concern here.*
** Set-Up and Using the Code
Open your terminal, making sure you are in the directory you want the repository
cloned to.
*The Bash (/.sh/) scripts need calling before the Python (/.py/) ones.* You need to
process the =lm1-exhibition-all.csv= and =lm2-exhibition-all.csv= file first because
the Python (/.py/) scripts assumes certain files are already in the =/data=
*The Bash (.sh) scripts need calling before the Python (.py) ones.* You need to
process the =lm1-exhibition-all.csv= and =lm2-exhibition-all.csv= files first because
the Python (.py) scripts assumes certain files are already in the =/data=
directory.
#+begin_src shell
@ -143,8 +144,8 @@ directory.
#+end_src
When the packages have finished installed (via ~pip~), you should be ready to
go. From there, you can simply call the Python (/.py/) scripts (from terminal with
the venv. activated). For example,
go. From there, you can simply call the Python (.py) scripts (from terminal with
the =venv= activated). For example,
#+begin_src shell
# Make sure you are in the terminal with the virtual-environment activated...
@ -155,22 +156,30 @@ the venv. activated). For example,
# Output from script...
#+end_src
When you have finished, you will need to deactivate your Python
virtual-environment. You can do that by entering ~disable~ in your terminal. You
should see the ~(venv)~ part of your prompt removed.
When you have finished, you will need to deactivate the virtual-environment. You
can do that by entering ~disable~ in your terminal. You should see the ~(venv)~
part of your prompt removed.
#+begin_src shell
# Before you disable your Python virtual-environment (venv)...
# Before you disable your Python's virtual-environment (venv)...
# (venv) yourname@yourpc:~/local-dev/ritherdon-charts$
# Disable your Python venv.
disable
# After you have disabled your Python virtual-environment (venv)...
# After you have disabled your Python's virtual-environment (venv)...
# yourname@yourpc:~/local-dev/ritherdon-charts$
#+end_src
** Working with the Files/Data Produced After Running the Project's Scripts
From here, you can either write you own scripts to form new charts or just play
with the CSV files in something like Microsoft Excel or Libre Office Calc.
** Working with the Files/Data Produced After Running the Project's Bash Scripts
#+begin_quote
=/data= stores the CSV files and =/output= stores the charts. Run ~./separator.sh~
to get started.
#+end_quote
When you clone the repository, you will find the =/data= directory will have the
following layout,
@ -188,9 +197,13 @@ data
0 directories, 2 files
#+end_src
*The =/output= directory not exist before you run the scripts.*
This =/data= directory is responsible for storing the /raw/ data (I.E. the CSV
files). The charts, created via the Python (.py) scripts, reside in the
=/output= directory.
*The =/output= directory should not exist until you run ~./separator.sh~.*
After you run the Bash scripts (~seperator.sh~ and ~totalilator.sh~), you should see
After you run the Bash scripts (~./seperator.sh~ and ~./totalilator.sh~), you should see
something like the following in the =/data= directory,
#+begin_src shell :results code
@ -201,12 +214,12 @@ something like the following in the =/data= directory,
#+begin_src shell
data
├── light-meter-1
   ├── 2021-06-13 # Directory of readings taken per hour taken on 13/06/2021.
   ├── 2021-06-13 # Directory of readings taken per hour taken on 13/06/2021.
   ├── 2021-06-13.csv # File containing all the reading taken on 13th June 2021.
   ├── 2021-06-14 # Directory of readings taken per hour taken on 14/06/2021.
   ├── 2021-06-14 # Directory of readings taken per hour taken on 14/06/2021.
   ├── 2021-06-14.csv # File containing all the reading taken on 14th June 2021.
# More files and folder here...
   ├── 2021-07-30 # Directory of readings taken per hour taken on 30/07/2021
   ├── 2021-07-30 # Directory of readings taken per hour taken on 30/07/2021
   └── 2021-07-30.csv # File containing all the reading taken on 30th July 2021.
├── light-meter-1-daily-totals.csv # Total number of readings recorded for each day.
@ -215,12 +228,12 @@ something like the following in the =/data= directory,
   └── 2021-07 # Directory containing files with hourly totals (per day) for July.
├── light-meter-2
   ├── 2021-06-13 # Directory of readings taken per hour taken on 13/06/2021.
   ├── 2021-06-13 # Directory of readings taken per hour taken on 13/06/2021.
   ├── 2021-06-13.csv # File containing all the reading taken on 13th June 2021.
   ├── 2021-06-14 # Directory of readings taken per hour taken on 14/06/2021.
   ├── 2021-06-14 # Directory of readings taken per hour taken on 14/06/2021.
   ├── 2021-06-14.csv # File containing all the reading taken on 14th June 2021.
# More files and folders here...
   ├── 2021-07-30 # Directory of readings taken per hour taken on 30/07/2021.
   ├── 2021-07-30 # Directory of readings taken per hour taken on 30/07/2021.
   └── 2021-07-30.csv # File containing all the reading taken on 30th July 2021.
├── light-meter-2-daily-totals.csv # Total number of readings recorded for each day.
@ -235,7 +248,7 @@ something like the following in the =/data= directory,
#+end_src
*The =/output= directory should now exist in the project's root directory -- with
nothing in it.* This is where the charts will go, after running the Python (/.py/)
nothing in it.* This is where the charts will go, after running the Python (.py)
scripts. The ~separator.sh~ script created it. The project's root directory should
now look like the following,
@ -256,7 +269,7 @@ now look like the following,
├── LICENSE
├── lm1-hourly-totals.py
├── lm2-hourly-totals.py
├── output # This should now exist and charts go in here...
├── output # *NEW* This should now exist and charts go in here...
├── README.org
├── requirements.txt
├── separator.sh
@ -266,3 +279,15 @@ now look like the following,
4 directories, 13 files
#+end_src
From here, you can either create some charts with the Python (.py) scripts and
take a look at them in =/output= or open the CSV files in =/data= and inspect the
data.
*** Overview of the CSV files in =/data=
I need to write this...
*** Overview of Scripts and Charts
I need to write this...

Loading…
Cancel
Save