READ ME

Contents

• Introduction
• System Design
• Design objectives
• Architecture
• User interface
• Implementation
• User interface
• Server software
• Data maintenance
• Directory organisation
• External dependencies
• Installation

Introduction

This document provides a system description of the on-line (WWW) searchable implementation (data and code) of Jeremy Wilkinson’s Gazetteer and Bibliography of the Mines and Quarries of North Wales.

It is provided primarily for anyone who may have to maintain the data or the system as a whole and anyone who may have to port the system to a new server.

System Design

Design objectives

The following objectives were adopted:

• The user interface should be as simple and intuitive to use as posible.
• The user interface should be interactive in that alterations to queries should immediately update the displayed results.
• To allow stepwise refinement of a query, a change to one element of the query should not affect the other elements of the query.
• Data such as site or person names etc. should be cross referenced, automatically where possible.
• Data maintenance should be as simple as possible taking the above objectives into consideration.
• Client-server traffic (in terms of the number of request and response transactions) should be kept to a minimum.
• External dependencies should be keep to a minimum.

Architecture

System functionality is partitioned between:

• The user interface, implemented as a collection of web pages (HTML files) plus CSS, JavaScript, map marker images etc.
• The server software, implemented as CGI scripts and included modules.

Communication between these subsystems in the form of HTTP requests and responses. Each search request from the UI generates a single HTTP request and its response.

User interface

This is structured as:

• a search query form
• a response display area

The query form uses drop-down menus for the majority of search terms and the minimum of free text fields. The use of menus guides the user in formulating searches and restricts the amount of input validation required to just the two free text input fields. These fields are validated as follows:

‘name’ input field valid entries:

• site or person (or comapny etc.) names as text (spaces and punctuation are ignored) e.g. morfadu will match MORFA DU and MORFA-DU
• regular expression delimited by ‘/’ e.g. /hafod?t/ will match HAFODTY and HAFOTTY
• one or more numeric site identifiers (this is mainly for use for data maintenance – see Data maintenance below).
if no sites are found that match the query text an appropriate message is displayed in the response area

‘place’ field valid entries:

• alphanumberic British Ordnance Survey grid reference e.g. SH61701798
• numeric coordinate pair British Ordnance Survey grid reference e.g. 261706, 317989
• latitude, longitude e.g. 52.741474, -4.048794
• gazetteer site name (or leading part thereof) e.g. Hafotty
• place name (or leading part thereof) e.g. Llanaber
• postcode e.g. LL42 1AL
unrecognised input will evoke an appropriate error message adjacent to the ‘place’ field

Responses to user queries in the form of text or map images are displayed in a delineated response display area.

Any change to the data entered in the query form will invoke a new search request (i.e. there is no explicit ‘Search’ button or icon).

Implementation

User interface

This is a collection of HTML, CSS, JavaScript and map marker image files. For each type of query (site, entity and grant) there is an HTML file and a corresponding JavaScript file. A further JavaScript file contains common functions etc. and there is a common CSS file to control overall styling.

To meet the objective of allowing the user to refine a single element of their query (or change the display option) and see the results immediately whilst leaving the other elements of the query unchanged nn AJAX technique has been used (albeit not using XML).

The UI:

• validates free-text input fields
• formats a query for POSTing to the server software
• receives the response from the server
• displays the response to the user as text or markers on a map as specified by the response

The web browser window can be resized without affecting system functionality.

The UI has been tested and appears to operate correctly with Firefox, Google Chrome, Microsoft Edge, Opera and Safari web browsers on Apple Macinstosh and Safari on Apple iPad.

Server software

This is a collection of Perl CGI scripts and supporting Perl modules. Perl was chosen as the scripting language primarily because of the author’s familiarity with it.

The server software:

• receives a query from the UI as CGI arguments, either from the UI query settings or from links in previous responses
• searches the appropriate data file for matches
• formats matches for display by expanding them and, for gazetteer display, appending cross references, source references, notes etc.
• returns the response to the UI

Scripts are invoked once per user request and terminate once the request is satisfied.

Data Maintenace

Data files are plain text flat files which are searched sequentially. A database was not used on the grounds of simplicity of data maintenance and to reduce external dependencies.

The optional URL argument SHOW_IDS will invoke the following aids to data maintenance:

Directory organisation

Top-level directory Wilkinson/

Sub-directory Wilkinson/info/

Sub-directory Wilkinson/ui/

User interface: HTML, CSS, JavaScript, map marker images etc.

Sub-directory Wilkinson/cgi/

Perl 5 CGI scripts (.cgi) and their supporting Perl module files (.pm)

Identifiers in code generally have meaningful names. Comments are generally minimal.

CGI development has used Perl v5.018.004

Sub-directory  Wilkinson/data/

The design philosophy is:

• Files should be human-readable (and as far as possible human-understandable) and editable with a text editor.
• There should be no need to pre-process files.
• Errors in data should fail silently as far as the user is concerned. (The checking program checkData.cgi can be used to identify bad or questionable data.)

File content:

• Files have a header comment record indicating field order and lengths.
• Blank lines and lines with # as the first character (i.e. comments) are ignored.
• Comments are provided to indicate specific requirements and constraints for data.
• As a consideration to the user (and data maintainer), site and entity file records should ideally be ordered by alphabetical order of names in the data. For the convenience of data maintenance, reference, role, employment and production files should be sorted numerically. Other than that, record ordering is not generally significant.

Sub-directory Wilkinson/translations/

Used to expand various fixed-length codes in data files for county, parish, product etc. to human-readable text.

Sub-directory Wilkinson/utilities/

Data maintenance utilities (not available on public server)

Sub-directory Wilkinson/search/

External Dependencies

System

The main external dependency is the use of the Ordnance Survey’s Data Hub map service to source OS map tiles used for maps on which site locations can be shown.

The Leaflet open-source JavaScript library is used to implement map display and related functionality.

The jQuery JavaScript library is used to simplify the JavaScript code. A local copy of this is included in the Wilkinson/ui directory.

External data

The gazetteer reference data contain links to a number of external sites. In particular use is made of National Library of Scotland’s online historic OS maps. All external links open in a new window and their non-availablity does not affect the operation of the gazetteer.

Installation

An appropriate Web server and Perl interpreter installation is required. (The current public server uses Apache 2.4.46 (Unix) and Perl v5.016.) The system requires approximately 2.7 MByte of disk space.

The top-level directory Wilkinson/ should be put in a suitable location for the web server to serve the html pages and execute the CGI scripts. (The server configuration should allow execution of the CGI scripts in the Wilkinson/cgi/ directory.)

CGI scripts should have ‘execute’ permission set for the server process. (If in doubt use ’world’ to allow CGI execution by any process.)

An API key for the Ordnance Survey data is required. This is stored as a text string assigned to the variable $API_KEY as the first non-comment statement in Wilkinson/cgi/site.cgi