System description and installation instructions for
Wilkinson’s Gazetteer and Bibliography of the Mines and Quarries of Wales



This document provides a system description of the on-line (WWW) searchable implementation (data and code) of Jeremy Wilkinson’s Gazetteer and Bibliography of the Mines and Quarries of North Wales.

It is provided primarily for anyone who may have to maintain the data or the system as a whole or who may have to port the system to a new server.

System Design

Design objectives

The following objectives were adopted:


System functionality is partitioned between:

Communication between these subsystems in the form of HTTP requests and responses. Each search request from the UI generates a single HTTP request and its response.

User interface

This is structured as:

The query form uses drop-down menus for the majority of search terms and the minimum of free text fields. The use of menus guides the user in formulating searches and restricts the amount of input validation required to just the two free text input fields. These fields are validated as follows:

Responses to user queries in the form of text, map images or messages are displayed in a delineated response display area.

Any change to the data entered in the query form will invoke a new search request (i.e. there is no explicit ‘Search’ button or icon).


User interface

This is a collection of HTML, CSS, JavaScript and map marker image files. For each type of query (site, entity and grant) there is an HTML file and a corresponding JavaScript file. A further JavaScript file contains common functions etc. and there is a common CSS file to control overall styling.

To meet the objective of allowing the user to refine a single element of their query (or change the display option) and see the results immediately whilst leaving the other elements of the query unchanged an AJAX technique has been used (albeit not using XML).

The UI:

The web browser window can be resized without affecting system functionality.

The UI has been tested and appears to operate correctly with Firefox, Google Chrome, Microsoft Edge, Opera and Safari web browsers on Apple Macinstosh and Safari on Apple iPad.

Server software

This is a collection of Perl CGI scripts and supporting Perl modules. Perl was chosen as the scripting language primarily because of the author’s familiarity with it.

The server software:

Scripts are invoked once per user request and terminate once the request is satisfied.

Data Maintenance

Data files are plain text flat files which are searched sequentially. A database was not used on the grounds of simplicity of data maintenance and to reduce external dependencies.

The optional URL argument MAINT will invoke the following aids to data maintenance:

argument omitted,
no value assigned or

Site, entity or grant record ids are not shown in output.
MAINT=1The site, entity or grant record id is shown in output for each record displayed.
MAINT=2As MAINT=1 plus
site and entity ids are shown within records
a reference source id query field is enabled.

To allow the effects of additions/amendments to the data files to be easily checked, when MAINT is set to 1 or 2:

Directory organisation

Top-level directory Wilkinson/

index.htmlGazetteer ‘front page’ with an introduction to the system and link to the gazetteer search page. It includes acknowledgments and copyright information. The Change Log ‘last updated’ date should be updated when changes are made to the Change Log file.
README.htmlThis file

Sub-directory Wilkinson/info/

changeLog.htmlSystem change log. Date of most recent update of this file should be recorded in the system ‘front page’ file.
gazetteerData.htmlBackground information for users about the origin of, and subsequent changes to, the data used by the gazetteer.
listsOfMines.htmlInformation for users about the Home Office Lists of Mines.
siteMap.jpgImage file for map of the whole area and site locations covered by the gazetteer. Used on site front page.

Sub-directory Wilkinson/ui/

User interface: HTML, CSS, JavaScript, map marker images etc.

index.htmlGazetteer user interface ‘front page’
entity.htmlSearch form and results display for individuals, companies and organisations
entity.jsJavaScript for entity.html
grant.htmlSearch form and results display for crown grants
grant.jsJavaScript for grant.html
site.htmlSearch form and results display for mine and quarry searches
site.jsJavaScript for site.html
wilkinson.cssStyle sheet for above HTML files
wilkinson.jsCommon JavaScript functions for the above HTML files
jquery.jsjQuery JavaScript API library (renamed from jQuery 3.5.1.js)
Image files for site markers on map display

Sub-directory Wilkinson/cgi/

Perl 5 CGI scripts (.cgi) and their supporting Perl module files (.pm)

entity.cgiIndividual, company and organisation search
grant.cgiCrown grant search
site.cgiMine and quarry site search
token.cgiServer for time-limited authorisation tokens required for fetching Ordnance Survey map tiles used in the ‘display as map’ site view.
osgb.pmConversion routines for latitude-longitude to/from British OS national grid and alphanumeric grid references to/from numeric grid coordinate conversions. Used by site.cgi. The code used for conversions between GB OS grid and latitude/longitude and their supporting subroutines is directly derived from Toby Thurston’s Perl cpan package Copyright © Toby Thurston ( 2002–2013.
wilkinson.pmCommon functions for CGI scripts. This module contains file names and file record formats (used with Perl unpack() functions) for the above scripts.

These scripts extract data in response to the search criteria, collate references, generate cross-references and apply style directives to the data for display.

Identifiers in code generally have meaningful names. Comments are generally minimal.

CGI development has used Perl v5.018.004

Sub-directory  Wilkinson/data/

The design philosophy is:

File content:

ENTITY.datIndividual, company and organisation names – each entity has a 5-digit identifier
ENTITY_INFO.datIndividual and company references and notes, limited company dates of incorporation, dissolution etc., registered address(es) and main objects clause text
GRANT.datCrown grants – each grant has a 5-digit identifier
SITE.datMine and quarry sites – each site has a 5-digit identifier
EMPLOYMENT.datNumbers of men employed above and below ground at sites
British grid references for post codes and place names – used by site.cgi when searching for sites within a specified distance of a postcode area or named place
PRODUCTION.datProduction records for sites
REFERENCE.datBibliographical references for sites. When displayed, reference page numbers are de-duplicated, page ranges are elided correctly, overlapping and contained page number ranges are normalised and references are sorted for output.
ROLE.datAssociates entities with roles and years active at sites (e.g. agent) or in companies (e.g. company director)

Sub-directory Wilkinson/translations/

Used to expand various fixed-length case-sensitive codes in data files for county, parish, product etc. => human-readable text.

COUNTY.datCounty identifier (used in GRANT.dat and SITE.dat) => county name e.g. M translates to ‘Merioneth’
EVENT_NAME.datEntity event identifier (used in ENTITY_INFO.dat) => event name e.g. i translates to ‘incorporated’
GRANT_TYPE.datGrant type id (used in GRANT.dat) => grant type e.g. 3 translates to ‘Take Note’
NLS_MAP.datMap series identifier and sheet number (used in REFERENCE.dat) => NLS sheet file name and survey/revision and publication dates (used to create links to maps on NLS website)
PARISH.datParish identifier (used in GRANT.dat and SITE.dat) => parish name e.g. AM translates to ‘Llanaber’
PRODUCT.datProduct identifier (used in SITE.dat) => product name e.g. MN translates to ‘manganese’
ROLE_NAME.datRole identifier (used in ROLE.dat) => role name e.g. AG translates to ‘Agent’
SOURCE.datSource identifier (used in REFERENCE.dat) => source title e.g. 013 translates to ‘Inspector of Mines Reports’

Sub-directory Wilkinson/utilities/

Data maintenance utilities (not available on public server)

checkData.cgiError and consistency checks on gazetteer data. This checks for:
  • limited companies with no company information
  • site references that do not have a corresponding site record (as may happen if a site is deleted but its references are not deleted)
  • translation codes such as product id, role id etc. that do not have a corresponding translation
  • duplicate entity and reference records
  • incomplete entity names (indicated by [possible completion?] following entity name in ENTITY.dat)
  • missing page numbers etc. in references
  • unresolved ‘see’ and ‘see also’ references
  • individuals active for more than 45 years (which may indicate possible conflation of two or more individuals with the same name)
wilkinson.pmCommon functions for CGI utility scripts (alias to in Wilkinson/cgi/ directory). Used by checkData.cgi.

Sub-directory Wilkinson/search/

index.htmlRedirection page following directory reorganisations 9 May 2020

External Dependencies


The main external dependency is the use of the Ordnance Survey’s Data Hub map service to source OS map tiles used for maps on which site locations can be shown. The Leaflet open-source JavaScript library is used to implement map display and related functionality.

The jQuery JavaScript library is used to simplify the JavaScript code. A local copy of this is included in the Wilkinson/ui directory in order to isolate the system from changes to the public version of this library which might have undesirable side-effects on the system.

External data

The gazetteer entity and reference data contain links to a number of external sites. Particular use is made of National Library of Scotland’s online historic OS maps. All external links open in a new window. The non-availablity of an external site or individual page does not affect the operation of the gazetteer.


An appropriate Web server and Perl interpreter installation is required. (The current public server uses Apache 2.4.46 (Unix) and Perl v5.016.) The system requires approximately 3.8 MByte of disk space.

The top-level directory Wilkinson/ should be put in a suitable location for the web server to serve the html pages and execute the CGI scripts. (The server configuration should allow execution of the CGI scripts in the Wilkinson/cgi/ directory.)

CGI scripts should have ‘execute’ permission set for the server process. (If in doubt use ‘world’ to allow CGI execution by any process.)

An API key and associated ‘secret’ character string for the Ordnance Survey data is required. These is stored as const text strings in Wilkinson/cgi/token.cgi

Return to Gazetteer front page