One of the less pleasant consequences of moving from Echunga to Dereel in July 2007 was that I lost my ADSL line and had to resort to other methods. After a disastrous attempt to use Telstra's NextG service, I got a satellite connection with IPStar. This was the most unreliable network connection I have ever had. In December 2010, I moved to Internode's NodeMobile Data service.

This page bases on earlier pages that I wrote for the DSL and satellite connections.

The graphs show:

• Link status. This is the connectivity to the other end of the 3G link, measured by an ICMP echo (ping) packet once a minute. The value is a reciprocal function of the ping time, 0.5 / ping_time. Thus 5 represents a ping time of 100 ms, and 1 represents 500 ms. 0 indicates a link failure.

• Network status. This is the number of remote systems responding to a ICMP echo (ping) packet at the same time as the link status ping. I currently ping 5 systems which I frequently access: freefall.FreeBSD.org, w3.lemis.com, www.auug.org.au, ozlabs.org and ftp.NetBSD.org. Normally this value should be 5. If it's less, the script retries every second until full connectivity is restored.

• “TCP speed”, a reciprocal function of the time it takes to transfer a very small “web page” (four bytes) from my external web server. The values shown are reciprocal seconds: 0.5 is 2 seconds, 2 is 0.5 seconds, etc. This graph doesn't directly show the loss of TCP connectivity: as long as the “speed” is above 0, I have connectivity. When the connectivity goes away, the “speed” drops to 0.

  This speed was of particular interest with the satellite link. I'm not sure it makes much sense over 3G, but we'll see.

The graphs were made with gnuplot, and I'm not very happy with the smoothing. In particular, the right-hand side of smoothed graphs has too much influence, and it's easy to get the false impression that the link status or TCP speed have changed significantly in the last few minutes. If anybody can point me at a way to fix this problem, I'd be grateful.

Click on the graphs for a 1600x1200 version.

Detailed outage information

With the satellite link I had continual outages, 1685 over a period of 3 years, or one every 15 hours. This section will monitor the 3G link for similar outages, though I hope they'll be much better. I have a program that evaluates the outage information and produces detailed outage information. Here's the information on the past 10 outages, along with the overall statistics:

Start time End time  Duration   Badness        from                    to
                     (seconds)

1298853133 1298853166     33	  2.019	# 28 February 2011 11:32:13 28 February 2011 11:32:46
1298855282 1298855306     24	  1.701	# 28 February 2011 12:08:02 28 February 2011 12:08:26
1298856691 1298856707     16	  2.599	# 28 February 2011 12:31:31 28 February 2011 12:31:47
1298881906 1298881972     66	  0.143	# 28 February 2011 19:31:46 28 February 2011 19:32:52
1298882887 1298882906     19	  3.934	# 28 February 2011 19:48:07 28 February 2011 19:48:26
1298883043 1298883064     21	 26.277	# 28 February 2011 19:50:43 28 February 2011 19:51:04
1299034769 1299034841     72	  0.024	#  2 March 2011 13:59:29  2 March 2011 14:00:41
1299036334 1299036604    270	  2.411	#  2 March 2011 14:25:34  2 March 2011 14:30:04
1299174976 1299175216    240	  0.026	#  4 March 2011 04:56:16  4 March 2011 05:00:16
1299572234 1299572387    153	  0.009	#  8 March 2011 19:17:14  8 March 2011 19:19:47

Total 167 outages, total time 241115 seconds (2 days, 18:58:35)
Average time between outages:	37888 seconds (10:31:28)
Average duration:		1443 seconds (00:24:03)
Availability:			96.19%
      

“Badness” is an attempt to quantify the effect. It's the reciprocal of the number of seconds per hour that the link was up between failures (i.e. 3600 / uptime).

And here is the summary information for the past 10 days with a less than 100% record:

Date        Outages   Duration  Availability    Date
                      (seconds)
1298293200	  2	   972	 98.88%	# 22 February 2011
1298379600	  2	  5226	 93.95%	# 23 February 2011
1298466000	 13	 25718	 70.23%	# 24 February 2011
1298552400	  6	 12881	 85.09%	# 25 February 2011
1298638800	  4	 30217	 65.03%	# 26 February 2011
1298725200	 18	  1728	 98.00%	# 27 February 2011
1298811600	 14	   348	 99.60%	# 28 February 2011
1298984400	  2	   342	 99.60%	#  2 March 2011
1299157200	  1	   240	 99.72%	#  4 March 2011
1299502800	  1	   153	 99.82%	#  8 March 2011
      

The dates in the left columns are in UNIX time_t format to ease further processing. I also have lists of all individual outages, or summaries per day.

The monitor scripts

The scripts that do this monitoring are:

• The script linkcheck performs the periodic pings to check link and network status. It writes the results to a log file for later processing.
• linkcheck invokes tcpcheck to check the TCP connection, since it could take forever. tcpcheck writes its own log file.
• The main script takes a raw gnuplot script and modifies it to create a total of 16 plots: four times (4 hours, 24 hours, one week and forever), two views (smoothed and unsmoothed) and two sizes (540x360 and 1600x1200). I chose 540x360 to fit two plots comfortably side-by-side on my screen; YMMV.

Download the scripts

I'm still adapting the scripts from the satellite versions. If you're interested, come back in a few days, or send me mail.