7.3 KiB
Summary Manchester
Setup Common Lisp Environment
You will not need to execute this code block if you've already set up SLIME in another ORG file. This is just in case this is the only file you're working on today, or it's your first file of the day.
Run m-x slime before running the following code. And, make note of the
:session attribute. It allows you to use the code in the code block to be use
in other code blocks which also use the :session attribute.
(ql:quickload :com.inuoe.jzon) ; JSON parser.
(ql:quickload :dexador) ; HTTP requests.
(ql:quickload :plump) ; HTML/XML parser.
(ql:quickload :lquery) ; HTML/DOM manipulation.
(ql:quickload :lparallel) ; Parallel programming.
(ql:quickload :cl-ppcre) ; RegEx. library.
(ql:quickload :plot/vega) ; Vega plotting library.
(ql:quickload :lisp-stat) ; Stat's library.
(ql:quickload :data-frame) ; Data frame library eqv. to Python's Numpy.
(ql:quickload :str) ; String library, expands on 'string' library.Rent Summary
I should point out here, the prices have deliberately removed overly expensive listings. They are unrealistic to my personal living conditions in 2024. So, the data relayed below is not a complete reflection of the companies/sources listed within it.
This table will be added to over time, as I work my way through more website/companies. I've decided to copy them over from the files holding said information. It's just easier to do so because of how small the data is. Coding this up would take to long, for little gain.
NOTE: ON THE MARKET LISTING DON'T CLEARLY STATE IF BILLS ARE INCLUDED IN THEIR RENT RATES. ASSUME THEY DON'T. See On The Move (Manc.) page for more context.
| Company | Quantity | Avg-Rent | Search-Range | Min-Rent | Max-Rent |
|---|---|---|---|---|---|
| Right Move | 121 | 1204 | £275–£1500 | 945 | 1495 |
| Spare Room | 647 | 589 | £100–£1260 | 300 | 1260 |
| Ideal Flatmate | 17 | 714 | £200–£1200 | 480 | 956 |
| Roomies | 16 | 641 | £0–£1200 | 400 | 1000 |
| On The Market | 438 | 1005 | £300–£1250 | 395 | 1250 |
| Open Rent | 114 | 735 | £400–£1500 | 400 | 1350 |
| Zoopla | 5 | 1138 | £400–£1300 | 1050 | 1250 |
| Prime Location | 109 | 1041 | £400–£1250 | 550 | 1250 |
(let ((filepath #P"working-data/manc-avg-rent-feb-march-2024.csv"))
(with-open-file (stream
filepath
:direction :output
:if-exists :supersede)
;; This needs to be added manually because of the use '|--+--|' in the table
;; above. If the table's header changes, you will need to updated here, as
;; well. It's a bit of pain but it is what it is.
(format stream "Company,Quantity,Avg-Rent,Search-Range,Min-Rent,Max-Rent~%")
(dolist (row table)
(format stream "~{~a~^,~}~%" row)))
(format t "[[file:~a]]" filepath))/craig.oates/overhaul2024/src/branch/master/working-data/manc-avg-rent-feb-march-2024.csv
(let* ((values
(lisp-stat:plist-df
`(:x-axis #("Right Move" "Spare Room" "Ideal Flatmate"
"Roomies" "On The Market" "Open Rent" "Zoopla"
"Prime Location")
:y-axis #(1204 589 714 641 1005 735 1138 1041)))))
(vega:defplot manc-avg-rent
`(:title "Monthly Average Rent Rates Manchester Feb–March 2024"
:mark :bar
:width 600
:height 600
:data ,values
:layer #((:mark (:type :bar)
:encoding (:x (:field :x-axis :title "Company" :type :nominal :axis ("labelAngle" 0))
:y (:field :y-axis :title "Rent (£)" :type :quantitative)
:tooltip (:field :y-axis)))
(:mark (:type :rule :color "darkorange" :size 3)
:encoding (:y (:field :y-axis :type :quantitative :aggregate :average)
:tooltip (:field :y-axis :type :quantitative :aggregate :average))))))
(vega:write-html manc-avg-rent "renders/manc-rent-averages-feb-march-2024.html"))/craig.oates/overhaul2024/src/branch/master/renders/manc-rent-averages-feb-march-2024.html
To save time, copying and pasting over the PNG file, just run this. Make sure
the HTML version of the chart saves the exported PNG file to your ~/Downloads/
directory.
mv ~/Downloads/visualization.png ./renders/manc-rent-averages-feb-march-2024.png
Website Listing Totals
I've created a data-frame here, instead of above, because this is a
work-in-programm file and I'm writing/working it out as I go along. I should
work this data-frame back into the manc-avg-rent plot above at some point.
(lisp-stat:defdf *man-sum*
(lisp-stat:read-csv #P"working-data/manc-avg-rent-feb-march-2024.csv"))#<DATA-FRAME:DATA-FRAME (7 observations of 6 variables)>
(vega:defplot man-sum
`(:title "Listing Amounts for Manchester (Feb. and March 2024)"
:width 600
:height 600
:data ,*man-sum*
:layer #((:mark (:type :bar)
:encoding (:x (:field :company :title "company" :type :nominal axis ("labelAngle" 0))
:y (:field :quantity :title "No. of listings" :type :quantitative)
:tooltip (:field :quantity)))
(:mark (:type rule :color "darkorange" :size 3)
:encoding (:y (:field :quantity :type :quantitative :aggregate :average)
:tooltip (:field :quantity :type :quantitative :aggregate :average))))))
(vega:write-html man-sum "renders/manc-listing-amounts-feb-march-2024.html")/craig.oates/overhaul2024/src/branch/master/renders/manc-listing-amounts-feb-march-2024.html
mv ~/Downloads/visualization.png ./renders/manc-rent-listing-amounts-feb-march-2024.png
Listing Amounts and Average Rent
Up to now (2024-03-19 Tue), Spare Room has the most listings and the lowest average rent. On top of that, the Spare Room listing include the bills in its rent rates.
Need to not draw too many conclusions until I've processed the other websites.