add robot.txt file to the /static directory.

I'm sure I will need to update this at some point. I didn't spend too much time
on it, just jotted things down as a I came across routes in web.lisp. The
Sitemap URL will need to change, the closer the site goes into production and an
domain name has been bought/finalised.

static/robot.txt

@@ -0,0 +1,28 @@
+# Nicola Ellis and Ritherdon Archive
+User-agent: *
+Sitemap: https://www.nicolaellisandritherdonarchive.com
+Disallow: /setup
+Disallow: /run-setup
+Disallow: /sign-up
+Disallow: /login
+Disallow: /logout
+Disallow: /dashboard
+Disallow: /site-settings
+Disallow: /site-settings/
+Disallow: /users
+Disallow: /user/
+Disallow: /create/
+Disallow: /user/
+Disallow: /edit/
+Disallow: /rename/
+Disallow: /page/
+Disallow: /storage/manage
+Disallow: /storage/rename/
+Disallow: /storage/delete/
+Disallow: /create/
+Disallow: /archive/
+Disallow: /search
+Allow: /archive
+Allow: /view/archive/
+Allow: /pages
+Allow: /view/page/