ritherdon-archive/README.org

* Nicola Ellis and Ritherdon Archive

  This is an website, written in Common Lisp and built on top of the Caveman2
  framework. Its main intention is to be the digital archive for the work
  produced by [[http://www.nicolaellis.com][Nicola Ellis]] during her time
  working alongside [[https://ritherdon.co.uk/][Ritherdon]].

  [[https://neandr.nicolaellis.com]]

* Overview of Technology Used

  Below is a **non-exhaustive** list of the tech. used to build and run this
  website:

  - [[https://lispcookbook.github.io/cl-cookbook/][Common Lisp]]
  - [[https://www.quicklisp.org/beta/][Quicklisp]]
  - [[Steel Bank Common Lisp][http://www.sbcl.org/]] (SBCL)
  - [[https://github.com/fukamachi/caveman][Caveman2]]
  - [[https://github.com/fukamachi/woo][woo]] (Common Lisp server behind Nginx)
  - [[https://github.com/fukamachi/mito][Mito]] (ORM)
  - [[https://github.com/fukamachi/sxql][SXQL]] (SQL Generator, used alongside Mito)
  - [[https://www.sqlite.org/index.html][SQLite3]]
  - [[https://github.com/mmontone/djula][Djula]] (Common Lisp port of the Django templating language)
  - [[https://www.debian.org/][Debian 11]]
  - [[https://www.nginx.com/][Nginx]]
  - [[https://www.meilisearch.com/][Meilisearch]] (Website's Search engine)

  For a complete list of packages used by Common Lisp, look at the
  [[/return-to-ritherdon/ritherdon-archive/src/branch/unstable/ritherdon-archive.asd][ritherdon-archive.asd]] file.

* System Overview

  The complete system is broken into two services:

  1. Ritherdon Archive (the main site)
  2. Meilisearch (the separate search system/service/instance)

  From an end-user's perspective, they shouldn't be able to tell the Meilisearch
  service is part of the overall system. When an end-user uses the Search
  features on the main site, the main site will make the requests to the
  Meilisearch service. From there, the Meilisearch service will return its
  results to the main site -- on the end-user's machine. You should see this
  closed-loop in the diagram below.
  
  #+begin_src mermaid
    graph TD
        Request((Request)) --> Server
        Server{Nginx} --> Archive
        Server -.-> Meilisearch
        Archive -. Search terms sent to server \n as their own requests .-> Request
        Request -.-> Server
        Meilisearch -. Results returned to Archive \n on clients machine .-> Archive
        Archive --> Response((Response))   
  #+end_src

  When it comes to the main site (Ritherdon Archive), it stores the archive data
  in the =/storage= directory and the SQLite3 database. Both are kept
  in-sync. by the main site -- as part of its feature-set.

  #+begin_src mermaid
    graph TD
        Request((Request \n /Response)) <--> Server
        Server{Nginx} <--> Archive
        Archive <-.-> DB[(Database)]
        Archive <-.-> Storage[/Files in /storage directory/]   
  #+end_src

* Installation

  To see what is being called, please read the [[/return-to-ritherdon/ritherdon-archive/src/branch/unstable/makefile][makefile]]. If you are unsure how
  to set-up an environment for developing in Common Lisp, please use the
  following link:

  - [[https://lispcookbook.github.io/cl-cookbook/getting-started.html][Common Lisp Cook Book: Getting Started]]
  - [[https://docs.meilisearch.com/learn/getting_started/quick_start.html#setup-and-installation][Meilisearch Doc's]] (Install Guide)

  Otherwise, just use the makefile.
    
  #+begin_src bash
    git clone https://git.abbether.net/return-to-ritherdon/ritherdon-archive.git
    cd ritherdon-archive
    sudo make install
    make lisp-install
    make quicklisp-add
    make search-install # Installs Meilisearch.
  #+end_src

 *Note:* The ~make search-install~ command adds the ~meilisearch~ binary to
 =/usr/bin/=. If you want to uninstall Meilisearch, you will need to delete it
  from =/usr/bin=. Run ~sudo rm /usr/bin/meilisearch~, to delete it.

* Run System on Local Machine

  Because the system consists of two systems running in tandem, you will need to
  have two terminals open to run them separately -- and see the logs for each
  service.

** Meilisearch

   Usually, you get this part of the system up and running before you get the
   main site working -- mostly because of the tasks flow easier -- but it's not
   essential to start this service first.

   #+begin_src bash
     # Make sure meilisearch is added to your /usr/bin/ directory.
     meilisearch --no-analytics
   #+end_src

   If you enter =http://localhost:7700= in your browser, you should see the
   meilisearch search page. Use =ctrl-c= to stop the service.

** Main Site (Ritherdon Archive)
   
  To run the main site (Ritherdon Archive), start Steel Bank Common Lisp (SBCL)
  by running ~rlwrap sbcl~ in your terminal. When SBCL has finished loading, run
  the following,

  #+begin_src common-lisp
    (ql:quicklisp :ritherdon-archive)
    (search:set-filter-attributes) ; Configures the Meilisearch service.
    (ritherdon-archive:start :server :woo)
  #+end_src

  Then go to =http://localhost:5000= in your browser.

  To stop the program, enter ~(ritherdon-archive:stop)~ in SBCL.

* Run System on (Prod.) Server

  *It is assumed you know how to register a domain name and point it your server
  running this website and the Meilisearch instance also.* I usually use a
  sub-domain of the main site (Ritherdon Archive) for the Meilisearch
  instance. For example, =https://www.nera.com= is the URL for the main site
  making =https://search.nera.com= the URL for the Meilisearch instance.
  
  This section builds on the one above (running on local machine). The main
  difference is getting the system to run on Nginx and as a Systemd service.

  The first thing to do is replace the parts which say ~<INSERT USERNAME HERE>~
  and ~<INSERT URL HERE>~ in the files in =/conf=. Those values are specific to
  your deployment environment. At the time of writing, there should be four
  files in =/conf=. They are:

  1. =meilisearch.conf= (the config. file for Nginx)
  2. =meilisearch.service= (the .service file for Systemd)
  3. =ritherdon-archive.conf= (the config. file for Nginx)
  4. =ritherdon-archive.conf= (the .service file for Systemd)

   After you have entered your details into the =.conf= and =.service= files,
   you can start to copy the files into their new locations.

** Set-Up Meilisearch
   
   #+begin_src bash
     # Nginx
     sudo cp ~/ritherdon-archive/conf/meilisearch-prod.conf \
          /etc/nginx/sites-available/meilisearch-prod.conf
     sudo ln -s /etc/nginx/sites-available/meilisearch.conf \
          /etc/nginx/sites-enabled/
     sudo systemctl restart nginx.service

     #Systemd
     sudo cp ~/ritherdon-archive/conf/meilisearch.service \
          /etc/systemd/system/meilisearch.service
     sudo systemctl daemon-reload
     sudo systemctl enable meilisearch.service
     sudo systemctl status meilisearch.service
   #+end_src

** Set-Up Ritherdon Archive (Main Site)

   #+begin_src bash
     # Nginx
     sudo cp ~/ritherdon-archive/conf/ritherdon-archive.conf \
          /etc/nginx/sites-available/ritherdon-archive.conf
     sudo ln -s /etc/nginx/sites-available/ritherdon-archive.conf \
          /etc/nginx/sites-enabled/
     sudo systemctl restart nginx.service

     #Systemd
     sudo cp ~/ritherdon-archive/conf/ritherdon-archive.service \
          /etc/systemd/system/ritherdon-archive.service
     sudo systemctl daemon-reload
     sudo systemctl enable ritherdon-archive.service
     sudo systemctl status ritherdon-archive.service
   #+end_src

   After everything is set-up, head over to the site's live URL and complete the
   set-up form.

** Set-up Filter Attributes for Meilisearch Instance

   After the search and main site's are set-up, you should be ready to set the
   filter attributes used by Meilisearch. The quickest way to do that is to run
   the following commands in SBCL (~rlwrap sbcl~),

   #+begin_src common-lisp
     (ql:quickload :ritherdon-archive)
     (search:set-filter-attributes)
     (exit)
   #+end_src

   You might need to stop the ritherdon-archive Systemd service ~sudo systemctl
   stop ritherdon-archive.service~ to run the above commands. If that is the
   case, remember to start the Systemd service, ~sudo systemctl start
   ritherdon-achive.service~.

* A Note on Using Ritherdon Archive

  This README has focused only on the developer side of the project. This is
  deliberate. I just haven't had the time to write any documentation on that
  side of the site or is the budget there to do so. Hopefully, this will change
  but at the time of writing (Nov. 2022), there is nothing in the repo's wiki.