#+options: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline author:t #+options: broken-links:nil c:nil creator:nil d:(not "LOGBOOK") date:t e:t #+options: email:nil expand-links:t f:t inline:t num:t p:nil pri:nil prop:nil #+options: stat:t tags:t tasks:t tex:t timestamp:t title:t toc:t todo:t |:t #+title: Summary Manchester #+date: \today #+author: Craig Oates #+email: craig@craigoates.net #+language: en #+select_tags: export #+exclude_tags: noexport #+creator: Emacs 29.1.90 (Org mode 9.7-pre) #+cite_export: * Setup Common Lisp Environment You will not need to execute this code block if you've already set up SLIME in another ORG file. This is just in case this is the only file you're working on today, or it's your first file of the day. *Run ~m-x slime~ before running the following code.* And, make note of the ~:session~ attribute. It allows you to use the code in the code block to be use in other code blocks which also use the ~:session~ attribute. #+begin_src lisp :session :results silent (ql:quickload :com.inuoe.jzon) ; JSON parser. (ql:quickload :dexador) ; HTTP requests. (ql:quickload :plump) ; HTML/XML parser. (ql:quickload :lquery) ; HTML/DOM manipulation. (ql:quickload :lparallel) ; Parallel programming. (ql:quickload :cl-ppcre) ; RegEx. library. (ql:quickload :plot/vega) ; Vega plotting library. (ql:quickload :lisp-stat) ; Stat's library. (ql:quickload :data-frame) ; Data frame library eqv. to Python's Numpy. (ql:quickload :str) ; String library, expands on 'string' library. #+end_src * Rent Summary I should point out here, the prices have deliberately removed overly expensive listings. They are unrealistic to my personal living conditions in 2024. So, the data relayed below is not a complete reflection of the companies/sources listed within it. This table will be added to over time, as I work my way through more website/companies. I've decided to copy them over from the files holding said information. It's just easier to do so because of how small the data is. Coding this up would take to long, for little gain. *NOTE: ON THE MARKET LISTING DON'T CLEARLY STATE IF BILLS ARE INCLUDED IN THEIR RENT RATES. ASSUME THEY DON'T.* See [[file:./on-the-market-manchester.org][On The Move]] (Manc.) page for more context. #+NAME: manc-avg-rent | Company | Quantity | Avg-Rent | Search-Range | Min-Rent | Max-Rent | |----------------+----------+----------+--------------+----------+----------| | Right Move | 121 | 1204 | £275–£1500 | 945 | 1495 | | Spare Room | 647 | 589 | £100–£1260 | 300 | 1260 | | Ideal Flatmate | 17 | 714 | £200–£1200 | 480 | 956 | | Roomies | 16 | 641 | £0–£1200 | 400 | 1000 | | On The Market | 438 | 1005 | £300–£1250 | 395 | 1250 | | Open Rent | 114 | 735 | £400–£1500 | 400 | 1350 | | Zoopla | 5 | 1138 | £400–£1300 | 1050 | 1250 | | Prime Location | 109 | 1041 | £400–£1250 | 550 | 1250 | #+begin_src lisp :var table=manc-avg-rent :results output raw (let ((filepath #P"working-data/manc-avg-rent-feb-march-2024.csv")) (with-open-file (stream filepath :direction :output :if-exists :supersede) ;; This needs to be added manually because of the use '|--+--|' in the table ;; above. If the table's header changes, you will need to updated here, as ;; well. It's a bit of pain but it is what it is. (format stream "Company,Quantity,Avg-Rent,Search-Range,Min-Rent,Max-Rent~%") (dolist (row table) (format stream "~{~a~^,~}~%" row))) (format t "[[file:~a]]" filepath)) #+end_src #+RESULTS: [[file:working-data/manc-avg-rent-feb-march-2024.csv]] #+begin_src lisp :results file (let* ((values (lisp-stat:plist-df `(:x-axis #("Right Move" "Spare Room" "Ideal Flatmate" "Roomies" "On The Market" "Open Rent" "Zoopla" "Prime Location") :y-axis #(1204 589 714 641 1005 735 1138 1041))))) (vega:defplot manc-avg-rent `(:title "Monthly Average Rent Rates Manchester Feb–March 2024" :mark :bar :width 600 :height 600 :data ,values :layer #((:mark (:type :bar) :encoding (:x (:field :x-axis :title "Company" :type :nominal :axis ("labelAngle" 0)) :y (:field :y-axis :title "Rent (£)" :type :quantitative) :tooltip (:field :y-axis))) (:mark (:type :rule :color "darkorange" :size 3) :encoding (:y (:field :y-axis :type :quantitative :aggregate :average) :tooltip (:field :y-axis :type :quantitative :aggregate :average)))))) (vega:write-html manc-avg-rent "renders/manc-rent-averages-feb-march-2024.html")) #+end_src #+RESULTS: [[file:renders/manc-rent-averages-feb-march-2024.html]] To save time, copying and pasting over the PNG file, just run this. Make sure the HTML version of the chart saves the exported PNG file to your =~/Downloads/= directory. #+begin_src shell mv ~/Downloads/visualization.png ./renders/manc-rent-averages-feb-march-2024.png #+end_src #+RESULTS: [[file:./renders/manc-rent-averages-feb-march-2024.png]] * Website Listing Totals I've created a data-frame here, instead of above, because this is a work-in-programm file and I'm writing/working it out as I go along. I should work this data-frame back into the ~manc-avg-rent~ plot above at some point. #+begin_src lisp :session (lisp-stat:defdf *man-sum* (lisp-stat:read-csv #P"working-data/manc-avg-rent-feb-march-2024.csv")) #+end_src #+RESULTS: : # #+begin_src lisp :session :results file (vega:defplot man-sum `(:title "Listing Amounts for Manchester (Feb. and March 2024)" :width 600 :height 600 :data ,*man-sum* :layer #((:mark (:type :bar) :encoding (:x (:field :company :title "company" :type :nominal axis ("labelAngle" 0)) :y (:field :quantity :title "No. of listings" :type :quantitative) :tooltip (:field :quantity))) (:mark (:type rule :color "darkorange" :size 3) :encoding (:y (:field :quantity :type :quantitative :aggregate :average) :tooltip (:field :quantity :type :quantitative :aggregate :average)))))) (vega:write-html man-sum "renders/manc-listing-amounts-feb-march-2024.html") #+end_src #+RESULTS: [[file:renders/manc-listing-amounts-feb-march-2024.html]] #+begin_src shell mv ~/Downloads/visualization.png ./renders/manc-rent-listing-amounts-feb-march-2024.png #+end_src [[file:./renders/manc-rent-listing-amounts-feb-march-2024.png]] * Listing Amounts and Average Rent Up to now (2024-03-19 Tue), [[file:./spare-room-manchester.org][Spare Room]] has the most listings and the lowest average rent. On top of that, the Spare Room listing include the bills in its rent rates. *Need to not draw too many conclusions until I've processed the other websites.*