TASK Windows Edition
version 1.1.3
2014 Edition
Tidal Analysis Software Kit Developed by the Marine Data Products team at the UK National Oceanography Centre.
Please read this user guide thoroughly before using the TASK software.
TASK – Windows Edition
Table of Contents TABLE OF CONTENTS ......................................................................................................... 1 Warranty and Liability ............................................................................................................ 4 Software Use and Licensing .................................................................................................... 5 THE NATIONAL OCEANOGRAPHY CENTRE .......................................................................... 6 TIDES, ANALYSIS AND PREDICTION .................................................................................... 7 Tides ..................................................................................................................................... 7 Observing Sea Level ............................................................................................................. 7 Mean Sea Level .................................................................................................................... 7 Surge Level ........................................................................................................................... 8 Tidal Level ............................................................................................................................. 8 From Tide Gauge to Tide Table ............................................................................................ 9 Tidal Harmonics Constituents ............................................................................................ 11 Tidal Harmonic Analysis ..................................................................................................... 12 TASK – WINDOWS EDITION ............................................................................................. 15 Introduction .......................................................................................................................... 15 Before Using the Software ................................................................................................. 15 Workflow ............................................................................................................................ 15 Data File Formats ............................................................................................................... 16 NOC CSV Format .............................................................................................................. 16 University of Hawaii – Sea Level Center Format ............................................................. 16 TASK‐2000 File Format ..................................................................................................... 18 Advisory Notes, Warnings and Errors ................................................................................ 19 INSTALLING THE SOFTWARE ............................................................................................ 20 Minimum Requirements .................................................................................................... 20 Installation .......................................................................................................................... 20 Where Files Are Installed ................................................................................................... 21 Commercial Licences .......................................................................................................... 21 STEP BY STEP HARMONIC ANALYSIS ................................................................................ 22 General comments before you start .................................................................................. 22 STEP 1: Start the Main TASK Application ........................................................................... 22 STEP 2: Using TASK‐Toolkit to create the TASK‐2000 file .................................................. 22 Setting the options for converting the data to TASK‐2000 format. ................................ 24 STEP 3: Looking at the Data with the TASK‐Graph Module ............................................... 30 Assessing Data Quality and Flagging Bad Data ................................................................ 33 Changing the Y‐Axis Scaling ............................................................................................. 35 STEP 4: Harmonic Analysis ................................................................................................. 36 Performing a Harmonic Analysis ...................................................................................... 36 Selecting the number of harmonics ................................................................................ 38 Page 1
TASK – Windows Edition
Analysing Short Time Periods of Data (less than 6 months). .......................................... 39 Looking Beyond the TASK Predictions ............................................................................. 39 Limitations of the TASK‐Analyse program ....................................................................... 39 Possible Warning / Error Messages ................................................................................. 40 STEP 5: Looking at the Analysis Output ............................................................................. 41 The Primary Results (PR) File ........................................................................................... 41 The Time Series (TS) Output File ...................................................................................... 45 STEP 6: Graphing the Data with the TASK‐Graph Module ................................................. 46 STEP 7: Deriving a prediction ............................................................................................. 47 Setting up TASK‐Analyse and POLTIPS‐3 Integration ....................................................... 48 Using TASK‐Analyse and POLTIPS‐3 Integration .............................................................. 48 REFERENCE SECTION ........................................................................................................ 50 Errors and Warnings ............................................................................................................. 50 TASK‐Toolkit ..................................................................................................................... 50 TASK‐Analyse ................................................................................................................... 52 Additional Automatic Checking of Results ........................................................................... 57 Daily and Monthly Means ..................................................................................................... 59 Submission of monthly means to PSMSL ........................................................................ 60 Handling Time Zones ............................................................................................................ 61 Daylight Saving Time ........................................................................................................ 61 Handling Datums .................................................................................................................. 61 Data Management and Quality Control Issues ..................................................................... 63 Data Management ........................................................................................................... 63 Quality Control ................................................................................................................. 63 The Data Manipulation Tool ................................................................................................. 64 What is TASK‐DMT used for? ........................................................................................... 64 Specifying the file ............................................................................................................. 64 Data Value Adjustment .................................................................................................... 65 Time Adjustment ............................................................................................................. 65 Flag Bad Data ................................................................................................................... 65 Splitting Files .................................................................................................................... 66 Channel swapping and comparison ................................................................................. 66 Using Other Software to Manipulate Data ........................................................................... 67 Analysis of Tidal Currents ..................................................................................................... 70 ADVANCED TIDAL ANALYSIS TOPICS ................................................................................ 71 How Daily Means Are Computed ....................................................................................... 71 Data Epochs and Historical Calendar System ..................................................................... 72 ‘Sa’ Phase Calculations in Different Software .................................................................... 72 Creating your own Harmonic Constant Sets ...................................................................... 74 Inferred Harmonics Revisited ............................................................................................. 75 S2 and K2 ........................................................................................................................... 76 Related / Inferred Harmonics .......................................................................................... 76 Example ........................................................................................................................... 76 Page 2
TASK – Windows Edition
Table of Equilibrium Tide Relationships .......................................................................... 77 Adding Inferred Harmonics to the HCSelect.txt file ........................................................ 77 Amplitude Adjustments ..................................................................................................... 78 APPENDICES .................................................................................................................... 79 Appendix A – Wave Terminology ....................................................................................... 79 Appendix B ‐ Glossary of Tidal Terms ................................................................................. 81 Appendix C1 – A Table of Standard Harmonics ................................................................. 83 Appendix C2 – Table of Standard Harmonics with Speed .................................................. 84 Appendix C3 – Day Numbers .............................................................................................. 85 Appendix D – Additional Reading Material ........................................................................ 87 Book List ........................................................................................................................... 87 Appendix E – Using the Microsoft Excel Graph Template ................................................. 88 Enabling Macros .............................................................................................................. 88 General Troubleshooting ................................................................................................. 89 Appendix F – Updating Your Dongle .................................................................................. 91 Extending your subscription / updating the dongle ........................................................ 91
Page 3
TASK – Windows Edition
Warranty and Liability 1. If the licence disk or CD‐ROM is defective NOC will replace it at no charge provided the defective item is returned within thirty (30) days from the date of despatch from the National Oceanography Centre (NOC). 2. NOC will refund the purchasing price provided all items (CD‐ROM, licence disk and user manual) is returned within thirty (30) days from the date of despatch from NOC and provided that all seals remain intact. No refund will be made if the seal has been broken on any items. 3. NOC will endeavour to contact all registered customers (by email) informing them that a new update is available for the software. This will be made available on CD or for download from the NOC FTP site. 4. NOC will provide basic technical support for all TASK ‐ Windows Edition users. This will be limited to installation problems only. 5. Technical support on issues relating to harmonic analysis, data processing and data quality control as well as support on using the various software modules will only be provided to users who have an appropriate support contract with NOC. 6. We do not guarantee that the functions contained in TASK ‐ Windows Edition will meet your requirements or that the operation of the program will be either error free or appear precisely as described in any documentation describing TASK ‐ Windows Edition. 7. No responsibility can be accepted by the UK Natural Environment Research Council (NERC) for any consequential loss or damage arising from any use that is made of TASK ‐ Windows Edition and output from the software howsoever caused. 8. All implied warranties, terms, representations and conditions, including (by way of example only) the implied conditions of satisfactory quality and fitness for any purpose of the information, are excluded to the fullest extent permitted by law. 9. This Agreement is governed by English law and the parties submit to the exclusive jurisdiction of the English courts. It is recommended that you only use this software if you have a good understanding of tidal harmonic analysis. If you are unsure of the processes involved in harmonic analysis or how to interpret tidal time series, you should contact the Tidal Analysis and Prediction Service at the UK National Oceanography Centre for advice. Although the National Oceanography Centre has done its best to make TASK – Windows Edition as user‐friendly as possible, it does not take away the requirement for the user being familiar with the process. It is a tool to help you in much the same way a hammer and saw is to a carpenter.
Page 4
TASK – Windows Edition
Software Use and Licensing The TASK – Windows Edition software is available under a range of licenses which permit the software to be used in different ways. In some cases, the different licenses will have restrictions imposed as to how the software can and can’t be used, and this may be enforced by one or more of the following: Different versions of the software containing hard‐wired restrictions / limitation, or having features disabled. The requirement to have a USB security dongle (supplied by NOC) present on the computer before the software can be run. The signing of a licence agreement which must be returned to the National Oceanography Centre before the software is made available. The different licences come under the following broad types: Commercial use (annual subscription) Academic use (for commissioned work) Academic use (for government funded blue‐skies research) Academic use (for teaching within a marine science or related programme) GLOSS / IOC / PSMSL related activities Marine Data Products Users are generally required to sign a licence and return this to National Oceanography Centre the Marine Data Products Team at the National Oceanography Joseph Proudman Building Centre at the address given in the box. 6 Brownlow Street Liverpool Use of the software within university departments for the L3 5DA purposes of teaching requires a separate licence to be signed United Kingdom. (“Use of TASK – Windows Edition software for teaching
[email protected] purposes”). Fax: +44 (0)151 795 4905 If you have any questions relating to the software licensing or terms and conditions of use, please contact the Marine Data Products Team who will be more than happy to answer any questions. An evaluation (demo) version of the software is available for evaluation purposes. This is restricted to generating a maximum of 2 harmonics (usually M2 and S2) from the data.
Page 5
TASK – Windows Edition
The National Oceanography Centre The National Oceanography Centre (NOC) is a component laboratory of the UK’s Natural Environment Research Council (NERC). The NOC has provided a national and international tidal analysis and prediction service since 1924, in fact the modern methods of tidal analysis and prediction used throughout the world are based on the work carried out by Dr. A.T. Doodson F.R.S. between 1919 and 1960 while he was working at the Liverpool Tidal Institute (the forerunner to the National Oceanography Centre). The NOC has continued to develop and refine the methods used to analyse tide gauge data and compute tidal levels and currents. Coupled with advances in modern computer hardware and software, we now have the most accurate tidal analysis and prediction software available. The NOC has offices in Liverpool (left) and Southampton (right), and combined they house a staff of around 500 scientists, engineers, IT and administrative support staff. It also houses various national and international facilities such as the British Oceanographic Data Centre (BODC), the Permanent Service for Mean Sea Level (PSMSL), the National Tidal and Sea Level Facility (NTSLF) and the British Ocean Sediment Core Research Facility (BOSCORF).
The Liverpool Site formerly known as the Proudman Oceanographic Laboratory
The Southampton Site
Page 6
TASK – Windows Edition
Tides, Analysis and Prediction
Tides Anyone who has spent more than a few days at the coast will be familiar with the regular rise and fall of the tide. If you were to measure the tidal level every hour for a whole year, you would notice some regular patterns in the rise and fall. For the majority of locations around the UK you would notice: the time between one high water and the next is, on average, around 12 hours 25 minutes – thereby giving two high waters and two low waters on most days, with the time of high water moving on about 50 minutes each day. often, the high waters alternate between a slightly higher high water and a slightly lower one. every 14 days, the tides appear to be slightly higher than average.
Observing Sea Level There are a large number of natural phenomena that cause the sea level to change. These include: Tides Surges Long‐term changes in the mean level Waves Seiches Tsunamis For most purposes, you can consider the level to be made up of the first three. Observed sea level = tidal level + surge level + mean sea level. Let’s briefly look at each of these influences on the observed sea level.
Mean Sea Level This is the average level of the tide calculated over a long period of time (usually a minimum of one year). This has generally increased globally by about 15cm over the past century due to the melting of ground ice (ice sheets and glaciers) and to the thermal expansion of the warming seas. There are some regional variations where the sea level appears to be falling; however, this is caused by the land rising faster than the sea.
Page 7
TASK – Windows Edition
Surge Level The surge level is the name given to the change in sea level caused by meteorological effects such as variation in the atmospheric pressure, prolonged onshore or offshore winds etc. As such, surge levels are very hard to predict accurately and require complex computer simulations. The UK National Oceanography Centre is a world leader in surge modelling, however even using some of the most powerful computers in the country, predicting a surge level more than a few days ahead is a futile exercise as the errors start to become significant and the accuracy drops off rapidly.
Tidal Level The tidal level is the change in sea level caused by the ocean’s response to the gravitational attraction of the Moon and the Sun. The tides are the predominant signal in a sea level record and are easy to distinguish from other components of sea level variation as they have such well defined periods. The tidal level is affected by numerous factors such as: the relative position of the three bodies in the Earth‐Moon‐Sun system. the variation in the Moon’s and Sun’s distance from Earth. the variation in the position (declination) of the Moon and the Sun. the local land masses and the shape of the coast. the depth of water at the location and surrounding areas. As you can see, the phenomena that affect tidal level are highly predictable (we can compute the position of the moon to an incredible accuracy), or change very slowly over time (coastlines rarely change overnight). Therefore it makes sense that tidal levels should also be predictable with a high level of accuracy and for dates well into the future. This is indeed the case. In this user guide, tidal level will often just be referred to as tide, and surge level as surge. It was discovered as far back as Isaac Newton (1642‐1727) that the individual influences on the tide can be modelled by summing up combinations of sine waves – a mathematical function very closely related to circular motion (or in our case, motion of water on a rotating Earth), however the modern harmonic method of analysing and computing the tide was developed by Arthur Thomas Doodson and published in 1921 when he was secretary of the Liverpool Tidal Institute (one of the many previous names for what is now the Liverpool site of the National Oceanography Centre).
Page 8
TASK – Windows Edition
From Tide Gauge to Tide Table The complete process of deriving a tide table showing the tides at any date and time into the future starts with having a good quality record of the actual tidal levels observed over as long a period as possible (ideally 18.6 years). The tide gauge records the vertical movements of the sea level at a regular interval (e.g. the height every 15 minutes).
Tide Gauge
Software: TASK ‐ Windows Edition
Quality control process
The data is carefully quality controlled to ensure any incorrect data is flagged.
Data reformatting & harmonic analysis
Derive a set of harmonic constants (a unique set of amplitudes and phases for each of the constituents).
Harmonic tidal calculation
Software: POLTIPS‐3
Use harmonics to derive the tide at any point in time.
Tide tables / tidal graphs etc. Reproduce the data in a ‘user‐friendly’ format.
The process starts with the installation of a tide gauge at the coast to record the variation in the sea level. To a casual observer, surface waves are the most obvious variation as they have a period measured in tens of seconds. The oscillations due to the tide have much longer periods (ranging from 30 minutes to over a day – with a few seasonal oscillations thrown in for good measure). These are sometimes not easy to observe due to the high frequency and amplitude of surface waves, therefore one of the principle jobs of a tide gauge is to remove this high frequency information (by applying a low‐ pass filter to the tidal signal). This can be done either physically or digitally on the tidal data. Traditionally a stilling well would have been used with an old‐ fashioned float gauge (in fact stilling wells are still sometimes used with modern acoustic and radar gauges). However, the Page 9
TASK – Windows Edition
UK’s National Tide Gauge Network developed and maintained by the National Oceanography Centre uses pressure transducers to record the sea‐level. Although the pressure transducer can be installed in a stilling well, the normal method of low‐pass filtering is carried out in the digital domain by sampling the tide at a 1 second interval and then averaging the data to 15 minute values.
Other types of tide gauge that have become very popular are the acoustic gauge and the radar gauge, primarily due to their relatively low cost. They are also quite easy to install and maintain, and usually come supplied with all the relevant data processing software. The output from these gauges will be compatible with the majority of data loggers. Once installed, they simply bounce a radar signal off the water’s surface to measure the height. For detailed information on the A radar gauge installed near Liverpool Docks, UK relative merits of each type of tide gauge, please refer to the Intergovernmental Oceanographic Commission report “Manual on Sea Level Measurement and Interpretation – volume 4” which is available for download from the Permanent Service for Mean Sea Level. (http://www.psmsl.org/train_and_info/training/manuals/). Once the sea level record has been recorded by the tide gauge it needs to undergo some quality control. For example the clock in the data logger may drift (or even stop completely) or the recording Page 10
TASK – Windows Edition
device by not be reporting the correct tidal height. The example below shows a sudden jump in the data for about 14 hours. This turned out to be a boat which someone had moored directly underneath a radar gauge – the radar, instead of bouncing off the surface of the water, bounced off the deck of the boat which was about 1.7m higher.
Data in the tide gauge record which is identified as wrong should either be fixed (if this is possible) or flagged as bad data so that it does not get used in the harmonic analysis. Following the quality control stage, the data can be harmonically analysed using a package such as TASK ‐ Windows Edition. This is used to generate a set of tidal harmonic constants unique to the location of the tide gauge.
Tidal Harmonics Constituents A tidal harmonic constituent is an individual sine wave that describes, either fully or in part, one of the numerous cycles that make up the overall variation in the tides. (See Appendix A for an explanation of the various terms related to waves.) To accurately predict the tides for a location you need to take into account all the different phenomena that influence the tide, and therefore many locations will make use of over 100 individual harmonic constituents to derive the resultant tide. These individual tidal cycles all have quite distinct frequencies related to the astronomical phenomena that influence the tides. For example: the rotation of the Earth relative to the Sun is 15.0000000°/hour the rotation of the Earth relative to the Moon is 14.4920521°/hour the variation in lunar distance is 0.5443737°/hour (a 27.5546 day cycle) Each of the distinct frequencies is related directly to a tidal harmonic constituent, or in some cases a pair of harmonic constituents which are close in frequency. These pairs of harmonics move in and out of phase to create a much slower oscillation as shown in the diagram below.
Page 11
TASK – Windows Edition
The result of adding two sine waves whose frequency differs by only 5% and where one has an amplitude of 2m and the other of 1m. When in phase they reinforce each other to give an amplitude of 3m. When out of phase, they partially cancel each other out to give an amplitude of 1m.
In the three examples in the bullet list on the previous page: the rotation of the Earth relative to the Sun is signified by a harmonic constituent called S1 the rotation of the Earth relative to the Moon is signified by a harmonic constituent called M1 the variation in lunar distance is constructed from a pair of harmonics , Q1 combined with M1. Many harmonics also have precise multiples which in many cases are far more significant in the mathematical reconstruction of the tide. For example, the harmonics M2 and S2 have frequencies exactly twice that of M1 and S1, but due to the semi‐diurnal (twice daily) nature of the tide, are far more significant (i.e. have a much greater amplitude than their diurnal counterparts). In the simplest terms, this is due to successive high and low tides generated by the Earth rotating beneath two ‘bulges’ of water (see diagram below) and therefore generating approximately two tides per day. So although M1 defines the speed of a whole rotation of the Earth relative to the moon (360° in 24 hours and 50 minutes = 1 lunar day), that single rotation will show two complete oscillations of the lunar tide, and therefore M2 (with a speed of 360° in 12 hours 25 minutes, or 28.9841042° per hour) is far more dominant than M1.
Tidal Harmonic Analysis Analysis involves reducing a set of sea level recordings from a tide gauge (usually at least 8760 hourly values covering one whole year) into a more manageable set of parameters which completely defines the tidal component of the tide gauge record. Since the tide is made up of a finite number of oscillations (sine waves) of fixed known frequencies, we can take this tide gauge data and compute the amplitudes and phases for each of the harmonic constituents using standard mathematical techniques. The amplitude is often given the mathematical symbol H, and the phase lag G. As the tidal influences change very slowly and the values are derived to a very high precision, these amplitude/phase pairs are often called harmonic constants. For a location with a long tidal record, we Page 12
TASK – Windows Edition
will usually derive up to 115 harmonic constants for each location (a complete list is given in the Appendix of this document).
The diagram above shows the energy density for a typical tidal record. You can see that there are some quite distinct spikes and these correspond to the frequencies of the known harmonics. The table below shows the 10 most significant harmonic constants for the Port of Liverpool. Constituent Speed (°/hr) Amplitude (H) Phase lag (G) (units: metres) (units: degrees) M2 28.9841042 3.334 181.3 S2 30.0000000 1.309 230.7 N2 28.4397295 0.654 164.4 K2 30.0821373 0.371 228.5 μ2 (mu2) 27.9682084 0.224 196.9 M4 57.9682084 0.193 298.5 MS4 58.9841042 0.148 354.7 L2 29.5284789 0.145 163.5 ν2 (nu2) 28.5125831 0.112 148.5 K1 15.0410686 0.093 96.8 O1 13.9430356 0.080 346.0 Once the set of harmonic constants is derived, they can be used with a tidal prediction program such as POLTIPS‐3 to compute the tide forwards or backwards in time for many years at a time if required. It is the high precision with which astronomical parameters (such as the speeds in the table above) are known that allows the tides to be computed so far in advance. Page 13
TASK – Windows Edition
Tidal prediction is basically the process of taking the harmonic constants derived from the analysis process and summing them together for the appropriate time. For the mathematically inclined, the equation is: cos
For more information on the mathematics involved, please refer to one of the books in Appendix D (Additional Reading Material).
Page 14
TASK – Windows Edition
TASK – Windows Edition
Introduction TASK ‐ Windows Edition is a suite of programs for Microsoft Windows based PCs that will enable you to take the data from a tide gauge, carry out the necessary data processing, compute a set of harmonic constants for that location and ultimately finish up with a fully formatted tide table for any date or time into the future. The modules included are: TASK‐Analyse The main harmonic analysis program. TASK‐Toolkit A program to help process and resample the data. POLTIPS‐3 A tidal prediction program designed to use custom sets of harmonic constants. TASK‐Graph An Excel template for looking at the time series data. TASK‐Loader A program used to launch the other programs and give access to help/tutorials.
Before Using the Software Before you use the TASK software, it would be advantageous to have at least a rudimentary understanding of tides and tidal analysis. The information in this user guide will go some way to providing this, but if there is anything you are unsure of, it is worth seeking clarification from another source (see Appendix D ‐ Additional Reading Material at the end of this user guide). It would also be useful to have some experience in manipulating data files. If you don’t have experience in computer programming, then a package such as Microsoft Excel 2007 (and any later version) will be capable of carrying out most of the data manipulation you might need to do. For advice and possible assistance in data processing, please send an e‐mail to the Marine Data Products Team at the UK National Oceanography Centre (
[email protected]).
Workflow The diagram on the right shows the basic workflow in carrying out a harmonic analysis starting with getting the data from the tide gauge through to producing a set of tidal harmonics, and ultimately through to the production of a printed tide table. Since much of the tidal analysis process is to do with manipulating data files, it will be useful to look at the different file formats before taking a more detailed look at the step by step process. Page 15
TASK – Windows Edition
Data File Formats There are three standard formats that are used by TASK ‐ Windows Edition for processing and manipulating data which are described below. The various formats need to be processed by TASK‐ Toolkit before the data can be analysed. This will convert the data into the TASK‐2000 format ‐ the input format for the TASK‐Analyse module. NOC CSV Format The default TASK‐Toolkit Input Format file is a simple comma separated values (CSV) file that is very easily created from any data manipulation or spreadsheet package such as Microsoft Excel. There is an optional number of header lines (which can contain any text) followed by the tide gauge data to be analysed. The order of values on the data lines is:
date time data1 data2 (optional) quality flag (0 or 1, optional) any additional information (optional, ignored)
Here’s a typical file (it has only one data value, a data quality flag and some other information which is always ignored by the TASK‐Toolkit processing software. Header line 1 Header line 2 01/01/2001,00:00:00,1.396,0,Anything here is ignored 01/01/2001,00:15:00,1.161,0 01/01/2001,00:30:00,1.066,0 01/01/2001,00:45:00,0.936,0 01/01/2001,01:00:00,5.791,1,This data point flagged as bad 01/01/2001,01:15:00,0.626,0 01/01/2001,01:30:00,0.501,0 Note that in the example above, the timing interval is the same (15 minutes), however this does not need to be the case with a NOC‐CSV file. TASK‐Toolkit will resample the data to a user‐specified fixed interval before outputting the TASK‐2000 file. It is advisable to use a .csv file extension for this type of file. University of Hawaii – Sea Level Center Format The University of Hawaii – Sea Level Center (UHSLC) serves multiple roles in supporting real‐time oceanographic operations as well as climate and oceanographic research. The UHSLC is an active contributor to the Intergovernmental Oceanographic Commission (IOC) Global Sea Level Observing System (GLOSS), and participates in operational and scientific oversight through the GLOSS Group of Page 16
TASK – Windows Edition
Experts. The UHSLC is primarily concerned with the implementation of the GLOSS Core Network of stations, the primary global observing system for in situ sea level. Data from the UHSLC global tide gauge network are supplemented by contributions of hourly data from agencies around the world on a monthly basis in support of the GLOSS Fast‐Delivery (FD) System. The rapid availability lends to limited quality control and calibration. The data support near real‐time oceanographic projects such as the Global Ocean Data Assimilation Experiment (GODAE) and preliminary comparisons for calibration of altimeters. You can visit the UHSLC at the following address: http://uhslc.soest.hawaii.edu/home Data can be directly downloaded from the UHSLC website which looks like this...
Saving the hourly CSV data will allow you to read it straight into TASK‐Toolkit and reformat it into TASK Format, as well as checking for any gaps in the data. The latitude, longitude and time zone of the data is not present in the CSV file however so you will need to also look at the information in the ‘Data’ column where the position and time zone appear on the first line of the file. Each line has the year, month, date, hour and sea‐level height (in millimetres). An example of the UHSLC CSV format is shown below: 2001,12,16,6,1333 2001,12,16,7,1130 2001,12,16,8,867 2001,12,16,9,553 2001,12,16,10,308 2001,12,16,11,177 2001,12,16,12,175 ... Page 17
TASK – Windows Edition
TASK-2000 File Format This is the native data file format for the TASK – Windows Edition software – all TASK users should read this section carefully and familiarise themselves with this format. All data should be converted to TASK‐2000 format before harmonic analysis using the TASK‐Toolkit module. One of the principle jobs of the TASK‐Toolkit module is to convert the data from one of the three accepted input formats into TASK‐ 2000 format prior to harmonic analysis. TASK‐2000 files are text files that consist of a number of header lines (which can contain any descriptive information relating to the data) and then one data record per line. Each data record consists of data corresponding to a single date and time as shown in the example below: Header line 1 … blank header line 20 (always 20 header lines in total) 1 0 2001 1 0.000 1.400 0.000 0.000 0.000 0.000 2 0 2001 1 0.250 1.160 0.000 0.000 0.000 0.000 3 0 2001 1 0.500 1.070 0.000 0.000 0.000 0.000 4 0 2001 1 0.750 0.940 0.000 0.000 0.000 0.000 5 0 2001 1 1.000 0.710 0.000 0.000 0.000 0.000
data channels must be 8 characters, see comment below on data channel spacing Each data record contains the following information:
Record number (sequential) Data quality flag (0 = good data ; 1 = bad data) Year (must be 4‐digits and no earlier than 1910) Day number (1 to 366 – table of day numbers given in Appendix C3) Time as a decimal hour (e.g. 3:45pm = 15.75) 5 data columns. These are referred to as channels 6 to 10 throughout this document
There are some strict requirements for the TASK‐2000 file format – some of the program modules require strict adherence to the format, whereas other parts allow some flexibility. Following these guidelines will ensure all programs in the TASK suite will work correctly.
Record numbers must be sequential and start at 1 – the software uses this as part of the data quality control. All modules have this requirement. If you use TASK‐Toolkit to generate the TASK‐2000 file, the record numbers will always be sequential beginning at 1. TASK‐DMT can be used to extract a subset of a TASK‐2000 and provides the option to auto renumber the records so that they always start from 1. The data quality flag should be 0 (for good data) or 1 (for bad/suspect data), however the TASK‐ Toolkit program will often generate a file with values of greater than 1 as advisory flags. These should be assessed and the analyst should make a decision as to whether these should be converted to a 0 or 1. However the TASK‐Analyse module does have the option to treat all non‐ zero flags as bad data. In this situation, all the non‐zero flags will be changed to a ‘1’ in the TASK‐Analyse ‘TS’ output file and thereby making it conform to strict TASK‐2000 format.
Page 18
TASK – Windows Edition
A strict TASK‐2000 file should have 20 headers lines, however TASK‐Toolkit and TASK‐Analyse will allow any number. If there are less than 20 header lines in the input file, TASK‐Toolkit will generate additional header lines to create a strict TASK‐2000 file. If there are more than 20 header lines in the input file, a warning will be given but all header lines will be copied. The time interval between successive lines of data should be identical throughout the file. Decimal time is rounded to 3 decimal places; therefore the data interval needs to be such that decimal rounding does not create an uneven spacing. (e.g. intervals of 6 minutes, 12 minutes and 15 minutes are permitted, but 10 minutes are not as the values would be 0.000, 0.167, 0.333 and this is not perfectly even due to rounding). Convention is that observation data goes in channel 6, and the predictions and residuals computed by TASK‐Analyse are put in channels 7 and 8 respectively. You must use this convention if you want to view the data in TASK‐Graph (the supplied Microsoft Excel based graphing module). It is also good practice to be consistent from one analysis to the next. TASK‐ Toolkit will always write any processed data to channel 6. TASK‐Analyse is flexible in the number of data channels on each line (it can handle from one to six data channels per line). The t2k file output by TASK‐Analyse will always have a fixed number of data channels on each line. TASK‐Analyse can handle flexible spacing of the data channels as long as there is a space or tab between each number. However to use TASK‐Graph, the spacing of the data channels must be exactly 8 characters each with at least one space before each number. Therefore for numbers greater than 999.999 two decimal places should be used instead of three (TASK‐Toolkit will automatically reduce the number of decimal places when converting CSV files if required). It is not uncommon for FORTRAN programmers to read/write numeric data with a format statement that does not leave a space between the numbers. This will cause problems with TASK‐Analyse.
1 0 2001 1 0.000 10.00010.50011.000 0.000 0.000
Advisory Notes, Warnings and Errors
Throughout the software and this user guide the following message levels are used: Advisory Note: This is usually a technical note about some aspect of the analysis or data processing. It is often just to provide a piece of useful advice or a recommendation to the analyst.
Warning:
A message informing the analyst that there may be an issue with the data, the processing or the software. These are usually not critical, and will not usually prevent the program from completing the task at hand.
Error:
This is usually more critical in that there is a problem with the data, and the program being used is unable to complete the task.
TASK ‐ Windows Edition will often make checks on the data in the background and only give a warning / error if it detects something that should be investigated further. A summary of all the advisory notes, warnings and errors that might be issued by the software is given in the Reference section of this user guide. Page 19
TASK – Windows Edition
Installing the Software
Minimum Requirements Operating System: Windows XP, Vista, Windows 7 or Windows 8 (The software may work on Windows 2000 but is untested) Hard Disk Space: 20MB for software ; recommended 50MB minimum workspace Screen Resolution: Minimum of 1280x1024 – any smaller can make using the application awkward Additional software: Microsoft Excel 2007 or later to use TASK‐Graph. Template was tested with Excel 2007. Note: Microsoft Excel is only required to use the Graph module – it is not needed for the harmonic analysis or using any of the other software tools.
Installation The software is normally supplied on CD which will contain the setup file. Insert the CD and double‐click on the setup file to run the software installation. You may get a warning message stating that the software is from an unknown publisher. Click on Yes to continue with the installation. Then work through the steps to install the software. You will need to accept the licence agreement before you can continue with the installation. Please read this before agreeing to it. It is recommended to accept the default installation folder for the files. The installation will create a TASK group on your start button with at least eight program shortcuts in it: TASK – Windows Edition The main program launcher TASK – Toolkit Main toolkit for format conversion and checking TASK – Data Manipulation Tool Data Manipulation Tool TASK – Analyse Harmonic Analysis module TASK – Graph Graphing Template for Microsoft Excel TASK – User Guide This document in PDF format POLTIPS – Tidal Predictions POLTIPS‐3 Tidal Prediction software POLTIPS – User Guide POLTIPS‐3 User Guide in PDF format Dongle Update Commercial/Academic version only Page 20
TASK – Windows Edition
Where Files Are Installed Due to the nature of the program in the way some of the programs update other parts of the program, and the way that different organisations have their systems locked‐down to allow user access only to specific parts of the system, it is beneficial to know how and where on your system the files have been installed. The main program files are, by default, installed into: C:\Program Files\National Oceanography Centre\TASK\ or C:\ Program Files (x86)\National Oceanography Centre\TASK\ if using a 64‐bit computer The POLTIPS software consists of numerous files and has therefore been installed in it’s own subfolder: C:\Program Files\National Oceanography Centre\TASK\poltips3\ or C:\ Program Files (x86)\National Oceanography Centre\TASK\poltips3\ The installation process will also create a folder in the user files area called: C:\Documents and Settings\\My Documents\TASK Data\ Windows XP or C:\Users\\Documents\TASK Data\ Windows Vista, 7, 8 In this folder will be placed the TASK‐Graph Microsoft Excel template (also available directly from the TASK group on the Start Button), and a subfolder called TestData containing some basic test data. When working on a harmonic analysis, we would recommend creating a subfolder under TASK Data for each analysis job that you carry out. In the following step‐by‐step section of the User Guide, the screenshots of the software are taken from different versions of the Microsoft Windows Operating system and therefore might not look exactly the same as they appear on your screen.
Commercial Licences If you purchased a commercial licence for TASK, you may have received a USB dongle (shown right). This must be inserted into a USB port on your computer before you can run the TASK‐Analyse module. Educational teaching and evaluation versions of TASK do not require a dongle, but are limited in terms of functionality. See Appendix F for details on renewing your subscription and updating the dongle.
Page 21
TASK – Windows Edition
Step By Step Harmonic Analysis This section of the user guide will work through a complete analysis. It will explain each step / program function in detail (in normal black text) as well as letting you know what you need to select for the sample data set included with the software (shown in blue boxed text like this). If you look in your ‘Documents’ folder you will see a folder called TASK Data, and in this a further folder called Test Data. This contains the following two files: TestData1.xls An Excel spreadsheet with 1½ months of data for a UK location. TestData1.csv A NOC‐CSV format file of the same data (saved directly from Microsoft Excel)
General comments before you start
Make sure your system meets the minimum requirements of the software, and that if you are using the standard version, you have the appropriate USB dongle plugged into your PC (if required). Familiarise yourself with the TASK‐2000 file format covered on page 18. Specifically take note of the spacing and limits on number size. If you want to use the POLTIPS‐3 software which is included in TASK, the data you are analysing must be in metres. If not, you can convert it into a TASK‐2000 file using TASK‐Toolkit, and then use the TASK‐DMT (Data Manipulation Tool) to change the units (to convert mm to metres, set the data value adjustment scaling to 0.001, and set it to process all records). Note that if use TASK‐Toolkit to convert a UHSLC data set into TASK format – the conversion from millimetres to metres is done automatically. Read the section on handling time zones and datums on page 61.
STEP 1: Start the Main TASK Application TASK – Windows Edition is actually a suite of individual programs. To make starting each of these easier, there is a module selection application (simply called TASK – Windows Edition.exe). Running this to start with will allow you to run any of the other TASK modules.
STEP 2: Using TASK-Toolkit to create the TASK-2000 file The first step is to convert the tide gauge data into a TASK‐2000 format file, and carry out some initial quality checking. To help achieve this, you can use TASK‐Toolkit, a collection of data manipulation and processing functions (the Program Function is selected from the drop down list at the top of the window). The main window for TASK‐Toolkit looks like this…
Page 22
TASK – Windows Edition
Run TASK‐Toolkit (either double‐click on TASK‐Toolkit.exe or use the Module Launcher Application) and under program function, make sure Convert from CSV to TASK format (with data processing) is selected (this is the default). The main jobs of this program function are:
Read in the input file. Generate the sequential record number. Convert the date and time into the year / day number / decimal hour. Resample the input data to the requested time interval so that the timing interval is perfectly consistent. If either of the ‘real’ data points either side of the resampled data point is flagged as bad, the resampled data point will also be flagged as bad. Check for any serious problems with the input data – such as time jumping backwards, or excessively large gaps in the input data. Check for data outside a specified range or for spikes in the data Either copy flags across or generate additional advisory flags to allow the analyst to see where potential problems may be. Generate the TASK‐2000 (.t2k) file.
Note that the first data value output by TASK‐Toolkit will always be on‐the‐hour, and therefore you may lose up to 59 minutes of data from the start of your file. Page 23
TASK – Windows Edition
Setting the options for converting the data to TASK-2000 format. 1. Select the input data file (TestData1.csv) by clicking the Browse button in the input options or drag‐ and‐drop the input file from an Explorer window to anywhere on the TASK‐Toolkit window. Select the appropriate format from the list (NOC CSV, UH‐SLC, TASK‐2000). TestData1.csv is a NOC‐CSV file. Note that when you select the input file (either using the browse button or by drag‐and‐drop, the output filename is also created for you. This will be the same name and the input file but with the filename extension changed to .t2k . If the input file is already a .t2k file, then _TT will be added to the filename before the .t2k . 2. Set the other input options 2.1. ‘Header Lines’ gives the number of lines in the file before the main data starts – these lines will either be copied to the output file or ignored completely. 2.2. Channel selection 1 channel only : only one data channel in the input file so use it Use ch.1 of 2 : two channels of data in input file ‐ use the first Use ch.2 of 2 : two channels of data in input file ‐ use the second 2.3. Flag present If checked, the data quality flag is copied to the output file. If not, all data is assumed to be good and given a flag value of zero. 2.4. All non‐zero flags ‐> 1 Allow the flag in the input file to be any numeric value. Any value other than zero will be converted to ‘1’. 2.5. Copy headers The header lines will be copied to the output file – additional headers will be added if required to make a total of 20 lines. For TestData1.csv there is only 1 header line before the data begins, 1 data channel and the data quality flags are present. Therefore enter ‘1’ for Lines to Skip, and ensure ‘1 channel only’ is selected and ‘Flag present’ is checked (ticked). 3. Set the output options 3.1. Filename Enter filename. It is advisable to use the same filename as the input file but with a .T2K extension instead of the .CSV one. This makes it easy to identify files that are in TASK‐2000 format. Note that the .t2k extension is not added by default if no extension is specified. 3.2. Flag Options When processing the data, TASK‐Toolkit will look for potential problems (for example by looking at gaps in the input data). The flag options are used to select how these are communicated to the user. These are called advisory flags as they neither indicate good or bad data, but should be looked at by the analyst. Page 24
TASK – Windows Edition
3.2.1. No change to input flags (other than resampling): No additional flags will be generated other than those required for the resampling process.
3.2.2. Advisory (advice flags set to 2, can overwrite a 1): Any potential causes for concern identified by the software will be flagged with a ‘2’ in the flag column. This will overwrite whatever value is there to start with (whether it is a 0 or 1).
3.2.3. Advisory (value indicates advice type) [default / RECOMMENDED]: Values greater than 1 can be added to the flag column. The number will indicate the type of warning. This is the recommended option for most analysis work.
1: Original data flagged as bad 2: Gap detection threshold – this data point falls between a gap that exceeds the threshold level set by the user. 4: This data points falls between a gap that exceeds the allowed variance from the mean. 8: Data range warning or spike warning. 9: Other warning or multiple warnings.
3.2.4. Advisory (with additional flag column): As for 3.2.3. but an additional column is added at the end of any line with ‘