This document provides a system description of the on-line (WWW) searchable implementation (data and code) of Jeremy Wilkinson’s Gazetteer and Bibliography of the Mines and Quarries of North Wales.
It is provided primarily for anyone who may have to maintain the data or the system as a whole and anyone who may have to port the system to a new server.
The following objectives were adopted:
System functionality is partitioned between:
Communication between these subsystems in the form of HTTP requests and responses. Each search request from the UI generates a single HTTP request and its response.
This is structured as:
The query form uses drop-down menus for the majority of search terms and the minimum of free text fields. The use of menus guides the user in formulating searches and restricts the amount of input validation required to just the two free text input fields. These fields are validated as follows:
‘name’ input field valid entries:
‘place’ field valid entries:
Responses to user queries in the form of text or map images are displayed in a delineated response display area.
Any change to the data entered in the query form will invoke a new search request (i.e. there is no explicit ‘Search’ button or icon).
To meet the objective of allowing the user to refine a single element of their query (or change the display option) and see the results immediately whilst leaving the other elements of the query unchanged nn AJAX technique has been used (albeit not using XML).
The web browser window can be resized without affecting system functionality.
The UI has been tested and appears to operate correctly with Firefox, Google Chrome, Microsoft Edge, Opera and Safari web browsers on Apple Macinstosh and Safari on Apple iPad.
This is a collection of Perl CGI scripts and supporting Perl modules. Perl was chosen as the scripting language primarily because of the author’s familiarity with it.
The server software:
Scripts are invoked once per user request and terminate once the request is satisfied.
Data files are plain text flat files which are searched sequentially. A database was not used on the grounds of simplicity of data maintenance and to reduce external dependencies.
The optional URL argument SHOW_IDS will invoke the following aids to data maintenance:
|SHOW_IDS=1||The site, entity or grant record id is shown in output (in red) for each record displayed. A ‘Refresh’ button is provided for each gazetteer record displayed so that changes to data can be made visible as soon as those changes have been made and the display refreshed.|
|SHOW_IDS=2||As SHOW_IDS=1 plus site and entity ids are shown (again in red) within records.|
|index.html||Gazetteer ‘front page’ with an introduction to the system and link to the gazetteer search page. It includes acknowledgments and copyright information.|
|changeLog.html||System change log|
|gazetteerData.html||Background information for users about the origin of, and subsequent changes to, the data used by the gazetteer.|
|index.html||Gazetteer user interface ‘front page’|
|entity.html||Search form and results display for individuals, companies and organisations|
|grant.html||Search form and results display for crown grants|
|site.html||Search form and results display for mine and quarry searches|
|wilkinson.css||Style sheet for above HTML files|
|Image files for site markers on map display|
Perl 5 CGI scripts (.cgi) and their supporting Perl module files (.pm)
|entity.cgi||Individual, company and organisation search|
|grant.cgi||Crown grant search|
|site.cgi||Mine and quarry site search|
|osgb.pm||Conversion routines for latitude/longitude to/from OS British national grid and alphanumberic numeric to/from numeric grid references. Used by site.cgi. The code used for conversions between GB OS grid and latitude/logitude and their supporting subroutines is directly derived from Toby Thurston’s Perl cpan OSGB.pm package Copyright © Toby Thurston (email@example.com) 2002–2013.|
|wilkinson.pm||Common functions for CGI scripts. This module contains file names and file record formats (used with unpack() expressions) for the above scripts.|
Identifiers in code generally have meaningful names. Comments are generally minimal.
CGI development has used Perl v5.018.004
The design philosophy is:
|ENTITY.dat||Individual, company and organisation names – each entity has a 5-digit identifier|
|SITE.dat||Mine and quarry sites – each site has a 5-digit identifier|
|EMPLOYMENT.dat||Numbers of men employed above and below ground at sites|
|LTDCO.dat||Limited company registered address(es) and main objects clause text|
|British grid references for post codes and place names – used by site.cgi when searching for sites within a specified distance of a postcode area or named place|
|PRODUCTION.dat||Production records for sites|
|REFERENCE.dat||Bibliographical references for sites. When displayed, reference page numbers are de-duplicated, page ranges are elided correctly, overlapping and contained page number ranges are normalised and references are sorted for output.|
|ROLE.dat||Associates entities with roles and years active at sites (e.g. agent) or in companies (e.g. company director)|
Used to expand various fixed-length codes in data files for county, parish, product etc. to human-readable text.
|COUNTY.dat||County identifier in GRANT.dat and SITE.dat to county name e.g. M translates to ‘Merioneth’|
|GRANT_TYPE.dat||Grant type id in GRANT.dat to grant type e.g. 3 translates to ‘take note’|
|NLS_MAP.dat||Map series identifier and sheet number in REFERENCE.dat to NLS sheet file name and survey/revision and publication dates (used to create links to maps on NLS website)|
|PARISH.dat||Parish identifier in GRANT.dat and SITE.dat to parish name e.g. AM translates to ‘Llanaber’|
|PRODUCT.dat||Product identifier in SITE.dat to product name e.g. MN translates to ‘manganese’|
|ROLE_NAME.dat||Role identifier in ROLE.dat to role name e.g. AG translates to ‘Agent’|
|SOURCE.dat||Source identifier in REFERENCE.dat to source title e.g. 013 translates to ‘Inspector of Mines Reports’|
Data maintenance utilities (not available on public server)
|checkData.cgi||Error and consistency checks on gazetteer data. This checks for:
|wilkinson.pm||Common functions for CGI utility scripts (alias to wilkinson.pm in Wilkinson/cgi/ directory)|
|index.html||Redirection page following directory reorganisations 9 May 2020|
The main external dependency is the use of the Ordnance Survey’s Data Hub map service to source OS map tiles used for maps on which site locations can be shown.
The gazetteer reference data contain links to a number of external sites. In particular use is made of National Library of Scotland’s online historic OS maps. All external links open in a new window and their non-availablity does not affect the operation of the gazetteer.
An appropriate Web server and Perl interpreter installation is required. (The current public server uses Apache 2.4.46 (Unix) and Perl v5.016.) The system requires approximately 2.7 MByte of disk space.
The top-level directory Wilkinson/ should be put in a suitable location for the web server to serve the html pages and execute the CGI scripts. (The server configuration should allow execution of the CGI scripts in the Wilkinson/cgi/ directory.)
CGI scripts should have ‘execute’ permission set for the server process. (If in doubt use ’world’ to allow CGI execution by any process.)
An API key for the Ordnance Survey data is required. This is stored as a text string assigned to the variable $API_KEY as the first non-comment statement in Wilkinson/cgi/site.cgi