Improving Recording and Reporting Traffic Net Statistics

C. Matthew Curtin, KD8TTE

Net Manager, Central Ohio Traffic Net

Sunday 5 February 2017

Summary

In the year 2016, Central Ohio Traffic Net operated for 114 hours, conducting 384 net sessions. With 4,023 stations checked in to those nets, we passed 1,448 messages.

In the course of reconciling our year-end results with monthly reports, we identified several types of errors. After performing an audit of the records and recomputing statistics we issued corrected reports for each of the twelve months of 2016. We have identified weaknesses in the reporting and tabulation system in use and have consequently adopted a new system that will result in greater transparency, accountability, and accuracy.

Introduction

The Central Ohio Traffic Net (COTN) exists “to relay traffic in the public interest during both normal times and communication emergencies and to train net members to accomplish this objective.”[1] Elected officials respond to emergencies by deploying resources available to them, and requesting resources from neighbors through their county, state, and federal emergency management agencies.[2] Those agencies strongly discourage self-deployment and in practice will make greatest use of resources only when relationships have been established before the emergency arrives.[3]

Recognizing this need to work with previously-established partners, FEMA and the American Radio Relay League (ARRL) announced a disaster communications partnership to provide resources including radio operators and technologists to support emergency operations.[4] In FEMA’s 2016 “Cascadia Rising” exercise, amateur radio traffic handlers provided a vital role in keeping the affected area in communication regionally, as well as between FEMA headquarters and the region.[5]

To ensure greatest utility in an emergency, COTN executed a Memorandum of Understanding (MOU) with Franklin County Emergency Management & Homeland Security (FCEM&HS) in 2015.[6] Furthermore, we have encouraged individual operators to secure FEMA qualifications in Incident Command System (ICS) and National Incident Management System (NIMS) that will help our volunteers to operate within the structure of response to an emergency. Finally, we have conducted exercises with FCEM&HS supporting both Light Search and Rescue and Damage Assessment operations.[7],[8]

As part of planning, emergency managers need to understand capability of their partners. While exercises are especially helpful for showing multiagency operation, they are relatively infrequent, while services like amateur radio relay operate year-round. ARRL encourages reporting station activity and uses such reports to show the capability and performance of amateur radio public service actually requiring it for Official Relay Stations[9] and Official Emergency Stations,[10] and publishing reports over a minimum threshold monthly in the League’s monthly magazine, QST. COTN’s operational capability and performance are of interest both as part of ARRL’s NTS and independently to its own served agencies such as FCEM&HS.

Methodology

We collected statistics for sessions throughout 2016, and performed reconciliation at the end of the year.

Collection of 2016 Statistics

COTN requires that operational statistics for sessions be reported by Net Control Stations to the Net Manager or a statistician.[11] As normally happens on traffic nets, such statistics are typically reported by radiogram addressed to the Net Manager, with text like

28 13 2 19 WA3EZN

Where the standard five-word radiogram indicates the day of the net, the number of stations checked in, the number of formal messages moved, the number of minutes in session, and the identification of any section net liaison (SNL) stations. Variations allow for multiple SNLs and sessions that operate at times other than the standard 19:15 local.[12]

COTN in turn reports to the Section Traffic Manager (STM) a compilation of the data into a monthly net report, summing the reported check-in, traffic, and minute counts, adding the number of sessions conducted for the month. Those in turn get compiled and are sent up to ARRL for its reporting.

In 2016, COTN had an appointed statistician who performed the function of taking session reports, compiling the results, and delivering the net report to the STM. COTN collected reports for all sessions and reported statistics for each month on the proper schedule for inclusion into the section report. On January 1, 2017, the statistician reported the results for the month of December and totals for the year.

Upon hearing the report for December and year-end, the net manager compiled the STM’s reports for COTN,[13] added the announced December result, and reconciled against the announced year-end totals.

Reconciliation

The Net Manager's calculation of year end totals showed differences from the statistician results.  Thus began the process of identifying the source of the problem.

The statistician reported raw data to the net manager, and the net manager performed monthly calculations based on the data. Results were different from the Section reports of COTN’s results. The statistician did not have his radiogram reports to the STM and could not get the data to agree with the published results. The net manager identified arithmetic errors in the addition of figures. The statistician went through his received session reports and identified data entry errors.

Ultimately the statistician provided raw data to the net manager based off his copy of the original net reports received from NCS stations.  

Results

The Net Manager’s tabulation of session statistics by month, as verified by independent tabulation by a representative of the COTN Advisory Board, is shown in Table 1.

The statistics vary from the figures reported to the Section Traffic Manager throughout 2016 in every one of the twelve reports submitted for 2016. We show which figures in each monthly report needed to be revised in Table 2.


Table 1: 2016 Corrected Statistics (Including Totals)

Checkins

Traffic

Time

Sessions

January

320

117

595

31

February

312

143

578

29

March

359

144

615

31

April

348

135

574

30

May

314

100

528

31

June

259

106

477

30

July

240

105

439

31

August

313

98

494

31

September

358

139

635

30

October

466

167

892

49

November

351

88

472

30

December

383

106

582

31

Total

4023

1448

6881

384

Table 2: Portions of Reports to be Corrected 

Checkins

Traffic

Time

Sessions

January

OK

Correction

Correction

OK

February

Correction

Correction

OK

OK

March

Correction

Correction

OK

OK

April

Correction

Correction

Correction

OK

May

Correction

OK

Correction

OK

June

Correction

Correction

Correction

OK

July

Correction

Correction

Correction

OK

August

Correction

OK

Correction

OK

September

Correction

Correction

Correction

OK

October

Correction

Correction

Correction

Correction

November

Correction

Correction

Correction

OK

December

Correction

OK

Correction

OK

Discussion

Restatement of statistics reported in a timely fashion to leadership and served agencies is unpleasant. We regret the errors and believe that we provide the most faithful service if we address them straightforwardly.

Marketing a Trustworthy Service

COTN’s objective to provide reliable radio relay of third-party messages is supported by its collection and report of activity. We believe that use of our service requires trust, which in turn requires transparency and reliability. We further believe that accountability is important for both the net control stations and COTN leadership tabulating and reporting the figures. Proper collection and report of our activity is critical to making the service usable: to be used, we must be known and trusted.

These principles are violated in conditions where one person has all session reports, and no one else in the organization is able to see the reports. In theory there was transparency by passing reports on-air as traffic in on a net session. In practice, such transparency requires another station or group of stations to record their own copies of the traffic as passed and at the end of the month be able to tabulate the figures and compare them to the net’s report as passed for the Section Traffic Manager.

Structurally there is neither incentive nor mechanism to provide the kind of verification needed to detect and to correct the errors that are certain to happen in the course of business even by well-intentioned operators.

Session Reporting and Tabulation

The 2014 remake of COTN’s web site included a form for submission of session reports. Report of session reports went by radiogram to the Net Manager or Statistician as had always been the case, leaving tabulation to be done privately. Ultimately, the Statistician chose not to use the web site feature, preferring to keep private local records as a matter of individual convenience. Having delegated the work, the Net Manager allowed the Statistician’s method.

Absent daily independent collection of statistics reported as traffic and tabulation of results, we had few opportunities to identify the problem. We missed one critical opportunity in the October results: the official results showed 31 sessions, even though the net ran a total of 19 sessions on one weekend in October (for SET) and convened every other night of the month. We might never have caught the problem had the Net Manager not independently tabulated totals for the year upon hearing the Statistician’s on-air announcement of December and year-end totals. In any case, the structure simply does not encourage verification of results.

New Reporting Procedure

Effective 25 January 2017, COTN has adopted a new reporting procedure to address these weaknesses structurally. Now, we require reporting through the online form. If a net control station for whatever reason cannot use the form, NCS will send a report to another station to enter the data using the form.  The form can be found on the left-side navigation bar on the link entitled “Submit a Session Report.”

The Net Manager (or designate) will then issue an acknowledgement in the form of a radiogram with ARL FIFTY THREE followed by the data as recorded through the web form.

The new procedure allows for NCS to see what has been recorded and to issue a correction if warranted, as well as to notice if the statistics function is not giving evidence of working properly.

In addition, the COTN web site has been given an additional page under “Reports”—called “Raw Session Statistics”—where the current year’s raw session reports are viewable to anyone who navigates to the page. Should any interested party wish to review what has been recorded for any reason, the underlying data are visible.

NCS use of the new procedure has shown the process to be easy while also supporting the goals of transparency.

Self-Critical Review

COTN is a volunteer organization. We give freely of our time and equipment to provide a public service and to train others to do the same.

We strive to conduct ourselves as professionals offering a reliable public service and to train others who wish to provide that service. As such, we conduct critical reviews of our performance in training exercises as well as in our operations. Mistakes are certain to happen. We will identify them and take corrective action, publishing our work in an effort to provide new volunteers a library of reference material to keep them from repeating mistakes we have already made, as well as to show our peers in public service what practices we have found to work best.

Readers taking this view of our report may find lessons they can apply in their own operations. We are always interested in feedback. The current lineup of COTN leadership can be found at www.cotn.us. The author of this report can be reached by radiogram:

C M CURTIN KD8TTE

BEXLEY OH 43209

614-398-0045


[1] Central Ohio Traffic Net. Articles of Association. Article 2.

[2] FEMA. National Incident Management System. https://www.fema.gov/national-incident-management-system

[3] FEMA. Developing and Managing Volunteers. IS-244. “Managing Spontaneous Volunteers.”

[4] FEMA. FEMA & ARRL Announce Disaster Communication Partnership. July 2014. https://www.fema.gov/news-release/2014/07/18/fema-arrl-announce-disaster-communication-partnership

[5] J. Wades. Designing an Emergency Communications Exercise Cascadia Rising and Amateur Radio. QNI Newsletter. Sep 2016. https://qninewsletterdotnet.files.wordpress.com/2016/09/qni-2016-9.pdf

[6] Franklin County Emergency Management & Homeland Security. Memorandum of Understanding. Executed __ 2015.

[7] C.M. Curtin. COTN Supports Franklin County Full-Scale Exercise. COTN News. Oct. 2015. http://www.cotn.us/news/cotnsupportsfranklincountyfull-scaleexercise

[8] C.M. Curtin. Central Ohio Traffic Net Report of Exercise and Improvement Plan. COTN Report. Nov. 2015. http://www.cotn.us/reports

[9] ARRL. Official Relay Station. http://www.arrl.org/official-relay-station

[10] ARRL. Official Emergency Station. http://www.arrl.org/official-emergency-station

[11] COTN. By-Laws. By-law 9. http://www.cotn.us/by-laws

[12] C.M. Curtin. How Do We Count Interrupted Net Sessions? COTN Training Tip. Aug 2016. http://www.cotn.us/tips/howdowecountinterruptednetsessions

[13] D. Maynard. From the Section Traffic Manager. ARRL Ohio Section Traffic Manager Web site. http://wa3ezn.blogspot.com/