craig.oates
/
overhaul2024
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

base: craig.oates:31c689ef6b1732cd8d98ebae07294a886f7eaf8c
Branches Tags
craig.oates:master
...
compare: craig.oates:cfb0408d7c93dac1083277b6e5175f1cf5ccf642
Branches Tags
craig.oates:master

2 Commits
31c689ef6b ... cfb0408d7c

Author SHA1 Message Date
Craig Oates cfb0408d7c create 2024-02-24-ideal-flatmate-manc.csv file.
2 months ago
Craig Oates 6857f75ad0 finish code for creating a CSV file for Ideal Flatmate.
2 months ago
2 changed files with 72 additions and 31 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats
  1. 84
      ideal-flatmate-manchester.org
  2. 19
      working-data/2024-02-24-ideal-flatmate-manc.csv

84
ideal-flatmate-manchester.org
Unescape View File

@ -84,48 +84,70 @@ wrong listings.
(incf counter)))))))
#+end_src
** TODO Create CSV of Listings
** Create CSV of Listings
Need to come back to this and finish it off. Left it in a mess because it's the
end of the day (at time of commit) and need to get some sleep.
#+begin_src lisp :results output raw
;; (with-open-file (out-stream
;; #P"working-data/2024-02-24-ideal-flatmate-manc.csv"
;; :direction :output
;; :if-exists :supersede)
;; (format out-stream "ROW-ID, OTHER STUFF")
#+begin_src lisp :results silent
(with-open-file (out-stream
#P"working-data/2024-02-24-ideal-flatmate-manc.csv"
:direction :output
:if-exists :supersede)
(let ((row-id 0))
(format t "|ROW-ID|LISTING-INFO|URL|~%")
(format t "|-|-|-|~%")
(format out-stream "ROW-ID,RAW-LISTING-INFO,RENT (£),LOCATION,URL~%")
(loop for file-path
in (directory #P"raw-data/external/2024-02-24_ideal-flatmate-manc-listings/*.html")
do (with-open-file (in-stream file-path)
(let* ((doc (plump:parse in-stream))
(listing (lquery:$ doc ".card-infos-left" (text)))
(cleaned-price (first (cl-ppcre:all-matches-as-strings "\\d+" (first (str:split "room" (aref listing 0))))))
(cleaned-location (str:replace-all "," "" (second (str:split "room" (aref listing 0)))))
(url (lquery:$ doc "a" (attr "href"))))
(format t "|~a|~a|~a|~%" row-id (aref listing 0) (aref url 0))))
(incf row-id)))
(format out-stream "~a,~s,~a,~a,~a~%"
row-id
(aref listing 0)
cleaned-price
cleaned-location
(aref url 0))))
(incf row-id))))
#+end_src
The amount of listings on Ideal Flatmate is tiny compared to the others. I can
basically show all them here.
#+begin_src shell :results output raw
csvlook working-data/2024-02-24-ideal-flatmate-manc.csv
#+end_src
#+RESULTS:
| ROW-ID | LISTING-INFO | URL |
|--------+---------------------------------------------------------------------+-------------------------------------------------------------------------|
| 0 | £690/month per roomChapel Street, Salford M3 5DZ, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id113377 |
| 1 | £740/month per roomChapel Street, Salford M3 5DZ, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id113378 |
| 2 | £841 - £842/month per roomMiddlewood Street, Salford, M5 4YW, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120130 |
| 3 | £746 - £750/month per roomSalford, M5 4ZF, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id122936 |
| 4 | £200/month 100, 100 Lloyd Mansions, Salford M6 6HA, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id122970 |
| 5 | £488/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id122929 |
| 6 | £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id123025 |
| 7 | £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id122962 |
| 8 | £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | https://www.idealflatmate.co.uk/auth/signup?f=b&uid=242365&pid=123025 |
| 9 | £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK | https://www.idealflatmate.co.uk/auth/signup?f=b&uid=210168&pid=122962 |
| 10 | £850/month per room7 Symphony Park, Manchester M1 7GB, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121033 |
| 11 | £850/month per room7 Symphony Park, Manchester M1 7GB, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121032 |
| 12 | £956 - £957/month per room7 Symphony Park, Manchester M1 7GB, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121034 |
| 13 | £980/month per room7 Symphony Park, Manchester M1 7GB, UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121030 |
| 14 | £678 - £679/month per roomSalford M5 4YW, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120131 |
| 15 | £708 - £709/month per roomSalford M5 4YW, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120127 |
| 16 | £725/month per roomSalford M5 4YW, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120128 |
| 17 | £775/month per roomMiddlewood Street, Salford, M5 4YW, UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120129 |
| ROW-ID | RAW-LISTING-INFO | RENT (£) | LOCATION | URL |
| ------ | ------------------------------------------------------------------- | -------- | -------------------------------------------- | ----------------------------------------------------------------------- |
| 0 | £690/month per roomChapel Street, Salford M3 5DZ, UK | 690 | Chapel Street Salford M3 5DZ UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id113377 |
| 1 | £740/month per roomChapel Street, Salford M3 5DZ, UK | 740 | Chapel Street Salford M3 5DZ UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id113378 |
| 2 | £841 - £842/month per roomMiddlewood Street, Salford, M5 4YW, UK | 841 | Middlewood Street Salford M5 4YW UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120130 |
| 3 | £746 - £750/month per roomSalford, M5 4ZF, UK | 746 | Salford M5 4ZF UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id122936 |
| 4 | £200/month 100, 100 Lloyd Mansions, Salford M6 6HA, UK | 200 | NIL | https://www.idealflatmate.co.uk/spare-room/salford/property-id122970 |
| 5 | £488/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | 488 | Joshua Grange Pluto Cl Salford M6 6HF UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id122929 |
| 6 | £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | 580 | Joshua Grange Pluto Cl Salford M6 6HF UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id123025 |
| 7 | £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK | 480 | Greater Manchester Manchester M31 4HZ 296 UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id122962 |
| 8 | £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK | 580 | Joshua Grange Pluto Cl Salford M6 6HF UK | https://www.idealflatmate.co.uk/auth/signup?f=b&uid=242365&pid=123025 |
| 9 | £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK | 480 | Greater Manchester Manchester M31 4HZ 296 UK | https://www.idealflatmate.co.uk/auth/signup?f=b&uid=210168&pid=122962 |
| 10 | £850/month per room7 Symphony Park, Manchester M1 7GB, UK | 850 | 7 Symphony Park Manchester M1 7GB UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121033 |
| 11 | £850/month per room7 Symphony Park, Manchester M1 7GB, UK | 850 | 7 Symphony Park Manchester M1 7GB UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121032 |
| 12 | £956 - £957/month per room7 Symphony Park, Manchester M1 7GB, UK | 956 | 7 Symphony Park Manchester M1 7GB UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121034 |
| 13 | £980/month per room7 Symphony Park, Manchester M1 7GB, UK | 980 | 7 Symphony Park Manchester M1 7GB UK | https://www.idealflatmate.co.uk/spare-room/manchester/property-id121030 |
| 14 | £678 - £679/month per roomSalford M5 4YW, UK | 678 | Salford M5 4YW UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120131 |
| 15 | £708 - £709/month per roomSalford M5 4YW, UK | 708 | Salford M5 4YW UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120127 |
| 16 | £725/month per roomSalford M5 4YW, UK | 725 | Salford M5 4YW UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120128 |
| 17 | £775/month per roomMiddlewood Street, Salford, M5 4YW, UK | 775 | Middlewood Street Salford M5 4YW UK | https://www.idealflatmate.co.uk/spare-room/salford/property-id120129 |
As you can see, all the entries are listed =/month=. So, I've not bothered to
create a /frequency/ column. I'm just going to move forward with that as the
default.
*Note:* There are a could of entries in the =URL= column which have valid URL's but
point to the incorrect page on the website. I'm just going to leave them in and
manually go to their listings page on the website. There isn't enough incorrect
URL's to fix this in code. On top of that, I'm more interested in the average
rent price per month than the actual locations right now.

19
working-data/2024-02-24-ideal-flatmate-manc.csv
Unescape View File

@ -0,0 +1,19 @@
ROW-ID,RAW-LISTING-INFO,RENT (£),LOCATION,URL
0,"£690/month per roomChapel Street, Salford M3 5DZ, UK",690,Chapel Street Salford M3 5DZ UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id113377
1,"£740/month per roomChapel Street, Salford M3 5DZ, UK",740,Chapel Street Salford M3 5DZ UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id113378
2,"£841 - £842/month per roomMiddlewood Street, Salford, M5 4YW, UK",841,Middlewood Street Salford M5 4YW UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id120130
3,"£746 - £750/month per roomSalford, M5 4ZF, UK",746,Salford M5 4ZF UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id122936
4,"£200/month 100, 100 Lloyd Mansions, Salford M6 6HA, UK",200,NIL,https://www.idealflatmate.co.uk/spare-room/salford/property-id122970
5,"£488/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK",488,Joshua Grange Pluto Cl Salford M6 6HF UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id122929
6,"£580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK",580,Joshua Grange Pluto Cl Salford M6 6HF UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id123025
7,"£480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK",480,Greater Manchester Manchester M31 4HZ 296 UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id122962
8,"£580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK",580,Joshua Grange Pluto Cl Salford M6 6HF UK,https://www.idealflatmate.co.uk/auth/signup?f=b&uid=242365&pid=123025
9,"£480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK",480,Greater Manchester Manchester M31 4HZ 296 UK,https://www.idealflatmate.co.uk/auth/signup?f=b&uid=210168&pid=122962
10,"£850/month per room7 Symphony Park, Manchester M1 7GB, UK",850,7 Symphony Park Manchester M1 7GB UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id121033
11,"£850/month per room7 Symphony Park, Manchester M1 7GB, UK",850,7 Symphony Park Manchester M1 7GB UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id121032
12,"£956 - £957/month per room7 Symphony Park, Manchester M1 7GB, UK",956,7 Symphony Park Manchester M1 7GB UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id121034
13,"£980/month per room7 Symphony Park, Manchester M1 7GB, UK",980,7 Symphony Park Manchester M1 7GB UK,https://www.idealflatmate.co.uk/spare-room/manchester/property-id121030
14,"£678 - £679/month per roomSalford M5 4YW, UK",678,Salford M5 4YW UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id120131
15,"£708 - £709/month per roomSalford M5 4YW, UK",708,Salford M5 4YW UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id120127
16,"£725/month per roomSalford M5 4YW, UK",725,Salford M5 4YW UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id120128
17,"£775/month per roomMiddlewood Street, Salford, M5 4YW, UK",775,Middlewood Street Salford M5 4YW UK,https://www.idealflatmate.co.uk/spare-room/salford/property-id120129
1 ROW-ID RAW-LISTING-INFO RENT (£) LOCATION URL
2 0 £690/month per roomChapel Street, Salford M3 5DZ, UK 690 Chapel Street Salford M3 5DZ UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id113377
3 1 £740/month per roomChapel Street, Salford M3 5DZ, UK 740 Chapel Street Salford M3 5DZ UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id113378
4 2 £841 - £842/month per roomMiddlewood Street, Salford, M5 4YW, UK 841 Middlewood Street Salford M5 4YW UK https://www.idealflatmate.co.uk/spare-room/salford/property-id120130
5 3 £746 - £750/month per roomSalford, M5 4ZF, UK 746 Salford M5 4ZF UK https://www.idealflatmate.co.uk/spare-room/salford/property-id122936
6 4 £200/month 100, 100 Lloyd Mansions, Salford M6 6HA, UK 200 NIL https://www.idealflatmate.co.uk/spare-room/salford/property-id122970
7 5 £488/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK 488 Joshua Grange Pluto Cl Salford M6 6HF UK https://www.idealflatmate.co.uk/spare-room/salford/property-id122929
8 6 £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK 580 Joshua Grange Pluto Cl Salford M6 6HF UK https://www.idealflatmate.co.uk/spare-room/salford/property-id123025
9 7 £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK 480 Greater Manchester Manchester M31 4HZ 296 UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id122962
10 8 £580/month per roomJoshua Grange, Pluto Cl, Salford M6 6HF, UK 580 Joshua Grange Pluto Cl Salford M6 6HF UK https://www.idealflatmate.co.uk/auth/signup?f=b&uid=242365&pid=123025
11 9 £480/month per roomGreater Manchester, Manchester, M31 4HZ, 296, UK 480 Greater Manchester Manchester M31 4HZ 296 UK https://www.idealflatmate.co.uk/auth/signup?f=b&uid=210168&pid=122962
12 10 £850/month per room7 Symphony Park, Manchester M1 7GB, UK 850 7 Symphony Park Manchester M1 7GB UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id121033
13 11 £850/month per room7 Symphony Park, Manchester M1 7GB, UK 850 7 Symphony Park Manchester M1 7GB UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id121032
14 12 £956 - £957/month per room7 Symphony Park, Manchester M1 7GB, UK 956 7 Symphony Park Manchester M1 7GB UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id121034
15 13 £980/month per room7 Symphony Park, Manchester M1 7GB, UK 980 7 Symphony Park Manchester M1 7GB UK https://www.idealflatmate.co.uk/spare-room/manchester/property-id121030
16 14 £678 - £679/month per roomSalford M5 4YW, UK 678 Salford M5 4YW UK https://www.idealflatmate.co.uk/spare-room/salford/property-id120131
17 15 £708 - £709/month per roomSalford M5 4YW, UK 708 Salford M5 4YW UK https://www.idealflatmate.co.uk/spare-room/salford/property-id120127
18 16 £725/month per roomSalford M5 4YW, UK 725 Salford M5 4YW UK https://www.idealflatmate.co.uk/spare-room/salford/property-id120128
19 17 £775/month per roomMiddlewood Street, Salford, M5 4YW, UK 775 Middlewood Street Salford M5 4YW UK https://www.idealflatmate.co.uk/spare-room/salford/property-id120129
Write Preview
Loading…
Cancel
Save