Enabling Communications, Anywhere, Anytime

The past year and a half we have put a large amount of effort into rebuilding the Serval Mesh application from the inside out, to include end-to-end encryption, integrated short-messaging and resilient file sharing, all made much, much more useful by a completely overhauled user interface.

Therefore it is with great pleasure that I announce our first release candidate for version 0.90 of the Serval Mesh.

This has only been possible due to the outstanding efforts, including during some difficult times, of our core team (in alphabetic order): Andrew Bettison, Romana Challans, Jeremy Lakeman and Corey Wallis.

To get an idea of the extent of changes, take a look at the release notes and related documents. Changes include:

If you have used version 0.08, you will notice these changes:

A completely redesigned human interface.
A much smaller APK; faster to download and install.
No need for third-party apps like SMSDroid or WebSMS.

The main screen now presents nine buttons:

Call to make voice calls
Messages to compose and view messages
Contacts to discover nearby phones on the Mesh and show your Contact List
Maps calls up the Serval Maps interface (if installed)
Share files to send files via the Rhizome file-distribution system, list and view received files, see how much storage you are using
Share Us to give the Serval Mesh software to other users with compatible Android devices
Settings to adjust settings (see below)
Switch Off(On) to stop or start Serval Mesh
Help for instructions and information

The help system is more detailed and complete:

Guide To Interface explains the buttons on the main screen
Accounts & Contacts explains how Serval Mesh identifies you and other users to each other
Licence is the full text of the software licence
Serval Security describes Serval's security features, Android permissions used, and the Privacy Policy
About introduces the Serval Project and leads to the Donate button
Quick Links contains some useful links for further reading
Serval Version is the full text of these release notes

The Settings menu has been overhauled:

Wifi Settings lets you examine and change Wi-Fi settings
Accounts Management lets you change your Serval Mesh phone number and name
View Logs shows a log of recent software activity
Redetect Wifi redetects the device's Wi-Fi chipset

There have been enormous changes under the hood:

The foundations of the Serval Security Framework are now in place. Elliptic curve cryptography is used for identifying, protecting and authenticating subscribers and mesh network traffic.
All Serval-to-Serval traffic (except Rhizome transfers) is now encapsulated in Serval's new, secure Mesh Datagram Protocol, implemented as an overlay network on standard IP over Wi-Fi.
The original Java implementation of the Rhizome file sharing system has been superseded by a new implementation in C within the serval-dna component, using SQLite as the local storage engine.
Voice calls are now carried over the mesh using Serval's own Voice over Mesh Protocol, which has been designed to replace SIPand RTP. As a result, call quality and latency have improved.
MeshMS (Serval's SMS-like service) now uses Rhizome as its transport.
Improved stability and responsiveness.

Version 0.90 is still considered experimental software, and there are a number of features and improvements that we intend to implement in releases between versions 0.90 and 1.00.

What we need now is for people to download and test this release candidate, and report any issues they encounter.

For more information, visit the Serval wiki page for this release candidate.

Update: see a video of us using 0.90RC1 here.

In a previous post, I began to discuss the Serval Technology stack, and gave a high-level overview of the lowest layers, upto the Serval Overlay Mesh (SOM), and the real-time Mesh Datagram Protocol (MDP), and protocols and services that sit above it. In this post I want to give an overview of the Rhizome store-and-forward or Delay Tolerant Networking (DTN) and the protocols and services that depend on it.

Rhizome sits over the Serval Overlay Mesh (SOM) layer, and uses the same public key identity system when required.

Rhizome itself is a bundle-based network protocol, where arbitrarily large bundles of data (often files, or compressed archives of files) are the basic unit of transport.

Each bundle of data is associated with a manifest which contains meta-data about the bundle, such as the name of the bundle, a SHA512 hash of the file so that reception of the file can be verified, and sender and recipient details where that makes sense.

Each manifest also contains a Bundle ID (BID), which is the public key in a CryptoSign key space. The BID is the unique identifier of the bundle. Each manifest is also signed using the private key corresponding to the BID, thus allowing detection and rejection of manifests that have been tampered with by a 3rd party.

It is also envisaged that manifests will later be signed by multiple keys, allowing filtering and grouping of bundles based on attesting authorities. This will be used as part of an overall system that will allow updating of Serval Mesh software via Rhizome over the Serval Mesh itself.

Where bundles are addressed to a particular party, the sender and recipient SID/ServalIDs (which are public keys of the sender and recipient, as discussed in a previous post on this blog.

This has the advantage that Diffie-Hellman style shared secret calculation can be performed to produce a cipher stream that can be XORed with the data in the bundle, ensuring that it can be encrypted/decrypted only by the intended recipient. The nature of a Diffie-Hellman shared secret calculation means that it can also be encrypted/decrypted by the sender, which provides repudiability, which is a valuable property for use in sensitive environments.

In a similar way, the private key corresponding to the BID can be stored in the manifest, allowing the recipient to update the manifest, e.g., to remove the file, once they have received it. This allows the bundle to be removed from the network. This ability to scrub stale content is an important property, because Rhizome is a store-and-forward based protocol, and thus each bundle may end up being replicated on every node in the network.

Rhizome itself provides a very flexible service that can be used to enable a wide variety of applications that depend on the resilient secure replication, and hence transport, of data from one node to another.

For example, MeshMS, which is our SMS-like service on the mesh uses Rhizome to distribute messages, even in the face of acute and chronic partitioning or islanding of the mesh. We have used this property in the past to send MeshMS from Africa to Australia, without depending on any telecommunications infrastructure.

We have also built, and are planning to build other services on top of Rhizome, such as voice mail (which is really just a variation on MeshMS), Serval Maps, our off-line collaborative mapping and situation awareness application, and of course general file distribution.

You can read more about Rhizome, MeshMS in:

Gardner-Stephen, Paul, Jeremy Lakeman, Romana Challans, Corey Wallis, Ariel Stulman, and Yoram Haddad. "MeshMS: Ad Hoc Data Transfer within Mesh Network." (2012)..

We hopped outside this week to run some tests on the Serval Mesh 0.90 release candidate to see how it performs outside, particularly Rhizome (including MeshMS) in a multi-hop configuration. We made the following video of the test.

What impacted me most in this testing, was how generally usable the new release is for general use. Provided the mesh could make the connection, phone calls were easy, and MeshMS and file sharing just worked. When coordinating the MeshMS tests, we used a mesh voice call to arrange things. Then when coordinating file sharing tests, we used MeshMS.

The photos we transferred in the video are shown below:

The photo I sent to Luke

The photo Luke sent me

This test also uncovered an issue with our multi-hop voice call routing, which need some work to prevent routing loops when there are many phones in a small area (we had six phones together in the basket on Romana's scooter).

From the outset the Serval Project has been using WiFi in mobile phones to form the mesh network, because that was the easiest radio to make use of in the phone.

However, WiFi has a lot of problems for meshing, especially on Android phones where you need to "root" the phone to even be able to use WiFi in the meshing ad-hoc mode. Ad-hoc WiFi also has a lot of interoperability issues, power consumption issues, as well as some scaling and mobility problems due to the relatively under-developed standard that underpins it.

But most of all, even if we solve all of those problems, WiFi has limited range. Typically mobile cannot communicate by WiFi over distances greater than 150m, and that's outdoors and without obstructions. Indoors it can be as bad as 10m, especially when there are walls and fences and other substantial obstructions.

This is why we have been thinking for a long time now about a pocket-sided "Mesh Helper" that will do the ad-hoc WiFi, so your phone doesn't need to be rooted, and won't go flat nearly so fast. But most importantly, we can put additional radios on the helper, so that we can form useful mesh networks over much greater distances.

A nice side effect of the mesh helper design is that anyone within WiFi range of the mesh helper can join the mesh, so you don't necessarily need one per person. Also, it means that people can keep on using whatever mobile phone they like, and upgrading it as often as they like, without worrying about mesh compatibility or losing the long-range communications capability.

We have some funding from the NLnet Foundation to explore prototyping the mesh helper device at present, so we have been working on the hardware and software needed to do this.

For a while we had our eyes on the HopeRF RFM23BP 1W ISM 915MHz band radio modules that are very small and cheap (about $10 in sample quantities), and fairly easy to program. That said, when you are talking about implementing the kind of frequency hopping required to use the ISM 915MHz band in Australia and the USA, "fairly easy to program" is a bit euphemistic for "not strictly impossible to achieve".

Fortunately, we have been following the progress of Andrew Tridgell (of Samba fame) with long-range telemetry radios for auto-piloting radio controlled model planes or UAVs. Andrew has been very patient with my frequent probing at Linux Conf AU (LCA) events where I have quizzed him on the current state of the radio equipment they are using, to determine the feasibility of using it to prototype our Mesh Helper device.

This year at LCA, it became clear that there was an excellent radio and firmware combination that would allow us to rapidly advance the prototype to a functional state. The only major trade-off would be that the prototype would feature a point-to-point radio link instead of a true mesh link, so only a pair of Mesh Helpers could operate in an area at a given time. This is totally acceptable for the prototyping stage. Also, the radio firmware has been largely written by Andrew, and is open-source, making it an excellent starting point for us to generalise the point-to-point link into a mesh link.

So we ordered a pair of the RFD900 radio modules from RFDesign, which arrived on Monday. A testament to the quality of the radios and firmware is that by yesterday afternoon we had Serval Mesh traffic flowing over them!

This was tested with a modified version of our servald software (currently living in the "serial" branch of github.com/servalproject/serval-dna, but soon to move into the "development" branch). The radios were plugged into my mac, and I had a separate instance of servald talk to each, and configured so that they could only communicate with each other via the radios, thus testing the full radio communication path.

After a few glitches with my SLIP-style encapsulation of packets in the serial data link, all was working, as the following video shows.

I did try to grab a couple of screenshots, but they seem to have disappeared into the ether, perhaps during one of the several times that my mac froze hard while talking to the radios. I need to look into the cause of that, which I suspect may be related to either the FTDI USB to TTL serial port driver or excess current draw by the radios over the USB port (however, the problem still occurred when transmitting at only +10dBm (10mW), which should have been well within the capability of the USB ports).

We also did a couple of indoor/urban tests with the radios. In both cases, a radio was left plugged into a mac either in my house just sitting on the lounge, or on a desk in our lab at work. We then wandered around with a 2nd computer with the other radio and kept an eye on the link quality to see how far away we could wander.

For the uban test, with the radio indoors, I then wandered outside and proceeded to walk around the block. Our block is about 75m wide and 200m long, with lots of solid brick homes and steel fences between houses. The radio was sitting only about 0.8m above ground level, well below fence level. The link was sustained almost the whole way around the block, up to about 150m - 200m through all the houses, walls, fences and other obstructions. In other words, if almost any two people on our block were using Mesh Helper devices, they could have easily communicated. In contrast with WiFi everyone on the block would have needed to be part of the mesh. This is exactly the kind of result we were hoping for. Also, when I got home I realised I didn't have the radios at full power, so it might even have been possible to get right around the block.

For the office test, we walked around inside the building from the 4th floor where our lab is, down to the 3rd and 2nd floors, all the way to the tea room (an important communications destination for us), and were able to maintain link most of the time, including while in the tearoom. It is worth noting that our building has thick concrete floors and is built into the side of a hill, so there were substantial barriers. We are not even certain if there is line of sight from the tea room to our lab above the soil of the hill side.

In the process of this, we have discovered that the full link budget (of up to about 125dB at typical noise levels) does not seem usable without some refinement of the forward error correction. This is something that we will look into when we have the opportunity, because it could potentially extend the maximum range in certain situations, where the loss of link is not due to complete obstruction of signal.

The full video of my recent talk from Linux Conf AU 2013 is now available:

http://mirror.linux.org.au/linux.conf.au/2013/mp4/Making_Mobile_Communications_Secure.mp4

The abstract for my talk is here: http://linux.conf.au/schedule/30058/view_talk?day=wednesday

"GSM/3G are surprisingly insecure, which is sad since good cryptographic frameworks exist. During the past year the Serval Project has been working on integrating very strong security into voice, text and data transfers on a mesh network. Rather than implement a secure SIP and secure RTP combination, we have taken a fresh approach and created a light-weight but secure packet and voice transport that is designed from the ground up with mesh networking in mind. One of the key innovations is using public keys as the network address, so that no key exchange or verification is required to setup an end-to-end encrypted channel. Consideration has also been given to how to defeat man-in-the-middle attacks for peers who are not able to verify each others keys prior to connection.

The system will be demonstrated in it's intended application in open-source Serval Mesh telephones to allow secure telephone calls.
Part of the talk will discuss the technical details of the security model, but (hopefully) in a fairly accessible manner that most developers should be able to follow, and in particular avoiding getting buried in mathematics. Feedback on the security model is invited so that any obvious vulnerabilities can be addressed before the software becomes widely distributed."

We finally have all the bits and pieces we need to assemble three prototype Serval Mesh Helper (SMH) devices, that under ideal conditions should be able to support links over tens of kilometres, instead of the tens of metres that WiFi is capable of.

The SMH has an embedded router (TP-LINK WR703Ns running OpenWRT -- see http://developer.servalproject.org/dokuwiki/doku.php?id=content:meshhelper:prototyping_on_mp2&#etc_config_wireless for more information) running the Serval Mesh software with 802.11n WiFi running in ad-hoc mode so that the SMH's can mesh with each other. They will simultaneously run in access-point mode so that phones can connect to them as WiFi clients, and hence not need to be rooted or otherwise able to do ad-hoc mode. This is the core functionality of the helper device.

The other capability in the helper is a long-range UHF radio operating in an appropriate ISM band, so that the mesh helpers can communicate over long distances. We are using RFD900 radios from rfdesign.com.au.

The fundamental components draw less than 1W sustained, and it can probably be optimised down to somewhere around 0.3W on average.

Here are some pictures of the assembly of these units.

First, for the power supply for the radio and RFD900s, we are using UBECs (little 5v switch-mode power supplies that are great for running from batteries). The RFD900s can be powered from USB to TTL serial cables (we are using FTDI ones at the moment), but at full power it is a bit too much for the WR703N's to pass through, so we are powering them separated.

This entailed soldering up a power connector that can plug into the UBEC and provide the correct power outlets for the WR703N and the RFD900 radio.

Then it was a case of plugging all the bits together. The following shot shows the connections to the radio modules:

Basically the black wires go on the left-hand edge, with the serial cable underneath, and the power connector on top.

We need a USB memory stick for extra storage (the WR703N has only 4MB flash, much too small for a Serval Rhizome node that might want to cache giga-bytes of data), which entailed adding a passive USB hub so that both the radio and memory stick could share the single USB port on the WR703N.

We are powering the components from large 120Wh LiFePO4 batteries, that should be able to run these units for close to a week. We opted for using car accessory power connectors as vehicle power is the most likely backup power source we expect to encounter:

One of the (many) nice things about the RFD900 radios is that they support antenna diversity. We take advantage of this by plugging in two antennae on perpendicular axes, so that we get horizontal and vertical polarisation (assuming we hang them vertically in the box). We also have some +6db yagis for when the need arises.

On the charging side of the batteries we used some nice connectors that we also fitted to our RedArc solar regulator, so that we can charge the batteries easily, including while using them to run the helper devices.

With such large batteries it is really important to make sure you have everything fused and protected against short-circuit, especially since we will be flying with these and don't want to accidently melt a hole in a plane.

Then it was time to think about a convenient container that would fit everything, be easy to carry around, and be weather proof enough for use at KiwiEx 2013 in a couple of weeks time. Some $10 storage tubs did the trick. We will put a small bag of rice in each when we take them on field to absorb any condensation in the tubs. You can see everything in there happily running:

Yesterday we setup a test link between two buildings here on the field exercise. The distance between the points was about 300m -- in theory short enough for a good WiFi link, like between a pair of Mesh Potatoes. However, mobile phone WiFi has poorer range for a variety of reasons. Also, there was 200m or so of damp forest in between, so any WiFi link would be likely to have problems. You can see the situation here:

A perfect chance to try out our prototype Mesh Helper devices with their UHF packet radios. Basically they look like this, with a OpenWRT router, UHF packet radio and a big LiFePO4 rechargeable battery in a plastic lunch bucket:

So we put one mesh helper at the wool-shed (the building on the left of the image above). The landing of the wool-shed was ideally situated, a bit elevated and facing towards the other building:

You can see the whole of the front of the wool-shed here. The large antenna mast is a HF radio link that is part of the exercise.

At the other end, we tried placing the other mesh helper on the roof of one of the out buildings where it was close enough to provide WiFi coverage for nearby phones, but hopefully provide a link to the wool-shed.

Unfortunately the roof was almost completely blocking signal, and so we didn't quite get link to the wool-shed from there. But by moving the mesh helper to the back of an adjacent building we got a strong signal (indicated by the RSSI and link-lock from the radios), although I forgot to write down exactly what the link budget was.

We then sent a message from the wool-shed back to the operations room near the other mesh helper device, which was successfully received:

What is nice is that all we have to do is to place the boxes, and everything connects automatically, keeping with our zero-configuration ideal for Serval Mesh networks.

The next step is to do some longer-range tests with one of the units on a good vantage point, so that we can get an idea of the maximum range possible with the current configuration.

After the short-range testing of the mesh helpers, we then set about creating a longer-range test so that we could determine the maximum useful range.

As part of the exercise, a VHF repeater was located on a prominent ridge, and this seemed like a good place to put a mesh helper to give us the chance to setup some long-range links.

The VHF repeater is on the left, and our mesh helper is hiding under the plastic bag on the right to keep out of the sun.

Here is a closer view of the mesh helper lashed to the fence post before receiving the plastic bag. You can get an idea of the kind of vantage point where the gear was placed.

We then proceeded to drive along the road towards Pongaroa with one of the mesh helpers in the car with us. Monitoring of link budget was using the built-in diagnostic webpage that we incorporated into servald.

That web page provides continuous link budget, calculated as difference between RSSI and noise-floor recalibrated into dB. I am not completely confident in the reading from this, but it is a good general guide.

For those interested, the radios we are using are transmitting at +30dBm, and using +3db antennae, that are admittedly poorly placed next to a large LiFePO4 battery and other gear in the buckets. Receiver sensitivity is around -121dB @ 1200bps, and so should be expected to be around -100dB @ 128kbps, giving a link budget of around 133dB.

Earlier, we had setup a second site nearer to the cottage, the results from which are marked as "site 2" in the following table. The primary site by the repeater is approximately 1km from the cottage, almost in the opposite direction from the road we travelled along, so the distances here are underestimates of the actual distances. The 2.65km and 2.68km records almost certainly correspond to actual distances of >3km.

The road varied in altitude, and had a lot of road-side vegetation and forest in places, as well as hills completely occluding line-of-sight to the sites where the other mesh helpers were located. This is reflected in the varying link budgets as we travelled along the road.

**Link Margin versus Distance**
Link Margin	Distance to Cottage	Notes
11dB	0.36km	Adjacent to the bridge
6dB	0.75km	road side
30dB to site 2	0.76km	road side
13dB to site 2	1.00km	road side
12dB	1.19km	On the hill side
18dB	1.33km	hill side
21dB	1.38km	track
10dB	1.48km	Side of road, next to a gate
2dB	1.63km	road side
13dB	2.00km	road side
23dB	2.00km	parked, antenna vertical
20dB to site 2	2.19km	road side
10dB	2.5km	While driving
22dB to site 2	2.65km	road
3dB to site 2	2.68km
14dB	26.02km	Steady 14db margin, but no lock/link.

It is nice to know that we could maintain a link over distances of around 3km. There were of course spots along the way where link was not possible due to vegetation, hills and other obstacles getting in the way.

Between 3km and the 26km there were hills blocking any line-of-sight, and so measurements were not possible there, except in one place at around 12km, where we could not detect signal.

3KM is way short of what we believe these radios to be capable of, as reflected by the strong link budget (22dB) at around that range. Certainly 12km should be plausible, if not possible.

We are exploring a number of possible reasons why we were not able to obtain links at these distances.

One of the main culprits we are exploring is the forward error correction on the radios. Currently they can detect and correct 25% BER in every 12 bits. However, there is no block code or convolution code to allow graceful handling of bursts of errors. The addition of such a code should allow links at greater distances. The 14db reading at 26km, which was steady over several minutes, offers tantalising hope that with improved error correction links over such distances may be possible. It is also possible that the number of black spots nearer might be reduced by such improvements in forward error correction.

Something like Turbo Codes would be ideal, but are too complex to implement in the radios, but could be implemented in the wireless router.

Another source of trouble is the prototype units themselves, where all the components are sitting together in a plastic bucket. There is almost certainly interference, occlusion and other problems occurring as a result.

It is nice to know that several kilometres is possible, and that there are some obvious areas we can work on to improve the range further.

What is significant is that the mesh helpers extend the range to useful distances, compared with the native WiFi capability of smartphones, as the following table summarising our experience shows.

Link Margin versus Distance
(with omnidirectional antennae)
Setting	SmartPhone WiFi	Mesh Helper
Urban, through dwellings	~20m	~200m
Vegetation	~20m	>300m
Rural	~200m	>3KM

Note that we are concentrating on omnidirectional antennae here, because our focus is on networks that are easy for normal people to setup, without having to aim antennae or have significant radio experience.

The Mesh Helper takes the range from being about one house to potentially tens of houses in distance, depending on the setting.

Thanks to the square rule, this means that instead of covering perhaps just one house, this means that we can cover potentially hundreds of dwellings with one unit, thus reducing the density of units required to establish a network -- which is precisely what we set out to achieve.

If it turns out that with improved forward error correction we can push those ranges out further, then the benefit will be even greater.

One of the next steps will be to reduce the size of the mesh helpers to mobile-phone size, to make them truly portable, which is entirely feasible as the core electronics are already small enough.

Where we have been staying in Moores Valley near Wellington, NZ, the mobile phone coverage drops out half-way along the road, and power and even the bridge are cut from time to time by fallen trees and the like -- all natural hazards of living in such a beautiful location.

Despite the isolation, it is easy to understand why people choose to live in places like Moores Valley

So for communities like this, there is real value in being able to easily make a backup communications system -- which is just what we designed the Mesh Helper concept to enable. By placing a few Mesh Helpers in homes, we can provide those houses with text and file communications amongst themselves, and possibly voice later on. With a bit of extra effort, we can then connect the local mesh to the outside world as well, but that's something we will work on later.

So I thought I would go for a wander up the road and see how far away I could get while still having a link.

Because I wanted the test to be reasonably realistic for this use-case, I just put one Mesh Helper on a book case in the patio here. The only special effort I made was about five seconds to generally point the antenna up the valley in the direction I was going to head -- and that is only necessary in these prototypes, because the antenna and large battery are sitting together in a bucket.

We could do much better by placing the Mesh Helper on a high point, such as on top of a building, but I really wanted to see how it would perform with the kind of placement that could be reasonably expected by non-technical people who just want to use it, without investing lots of time, money or energy.

I then proceeded to carry the other Mesh Helper up the valley road, and check the signal strength at every house number. As most people in the valley live on 10 acre sections, the houses are actually some way away from the road. So the results may not be 100% accurate. However, the road is generally lined with screening vegetation, which actually makes it a more challenging environment than adjacent to the houses themselves, which are mostly surrounded by paddocks.

Typical Road-Side Environment

Taking a measurement from on a letterbox along the road.

I plotted the results on a Google Map, and colour coded the measurement points according to signal strength, with blue being best, yellow least, and red meaning no signal:

https://maps.google.com/maps/ms?ie=UTF&msa=0&msid=202806144399311055643.0004d7472c2e45c1d3369

The result? We were able to obtain a link over distances of up to 1.27km, which is far enough to reach at least 13 houses in that direction, not counting those that might be reachable on the other side of the road, or on either side in the opposite direction along the road. This means there are perhaps 50 homes that can be reached from one site.

This is very important, because it means that a network can be constructed in a setting like this, even with only one out of 50 homes having a Mesh Helper.

Remember also that this is 1.27km between a pair of fairly casually placed Mesh Helpers.

With units placed in good vantage points (and some of these properties include areas close to the local ridge line) with clear line of sight, we already know that links of >3km are easily possible.

Also, once we finish rewriting the radio firmware, we will be able to mesh many of these Mesh Helpers, we will be able to create links many times the range of a single Mesh Helper link.

Thus it might well be possible to connect communities in rural, remote and developing settings over tens of kilometres.

A number of people now have asked whether a Serval Mesh can be connected to the cellular network, e.g., to allow people living just beyond the range of cellular service to be able to make and receive mobile telephone calls.

This would be best done using a cellular modem and a bit of software integration to make use of it, so that the cellular audio can be passed through digitally. However, most people who have asked about GSM/mesh gateways have indicated strongly that the solution must use only Android phones, and not require any soldering.

This is not unexpected, since it is hard to get new equipment into areas where disaster, rural, remote or poverty are at play. The same goes for areas impacted by war or unrest.

So we started thinking about how we can do this.

Unfortunately, most models of Android phone don't let you access the audio from the cellular call directly, which rules out the easy option of writing some clever software that can run on a phone.

However, all is not lost. We can use two phones, with one making the cellular end of the call, and the other handling the mesh end of the call. Then by using headsets to do some good old-fashioned acoustic coupling, we can record the cellular audio through the microphone of the mesh-connected phone, and play the audio out the speaker of the mesh-connected phone and into the microphone of the cellular-connected phone. A picture should help illustrate how this would be arranged in practice:

Yes, it really is that simple. And that ugly.

For extra simple and ugly, you could just tape the two phones together with one upside down so that the speaker and microphones are close enough to couple the audio.

Of course it has some drawbacks, notably the need for two phones instead of one, and the degredation in audio quality that is almost certain to occur. However, initial tests reveal that the audio quality is better than might be expected, and quite likely to be very usable.

Where we are up to now is that we have a student working on prototyping this. He has got to the point where a call from on the mesh can trigger an outbound call to the cellular network, and has the two ends of the audio playing. You can see this in the video below:

The next step, and the one that will get us to a complete usable prototype for mesh to cellular calls (the other way around is a bit trickier) is to make the Serval Mesh software know about wired headsets so that it can route the audio through that path.

Today I needed to charge our Mesh Helper prototypes (which we are thinking of calling Mesh Extenders instead -- anyone with suggestions for better names is invited to poke me with them). This is to get them ready for the Adelaide Mini Maker Faire

The trouble was I forgot to bring our mains charger for the LiFePO4 battery packs in the Extenders.

That left our solar charging system as the only way to charge the batteries. Fortunately this is South Australia, and we get a lot of good sunshine, even in Autumn. The declination of the sun is enough though, that finding a nice 30 degree angle for the solar panel to sit on is a good idea. Fortunately we have such a slope just outside the lab. It is even North-facing. So I hopped outside and setup our 40W panel with the two Mesh Extenders. They each have a 120W hour battery, so 240W hours total is needed, which should take about 6 hours, but the batteries are already partly charged, so they should be charged enough by the end of the day.

Here they are charging happily without relying on any fixed infrastructure at all -- not even for energy. A very Serval moment.

Stepping back after I took the photo, I realised that I had inadvertently captured a great contrast between infrastructure and the low-cost, portable resilience that we are championing: In the background of the photo you can see the 250KW generator (including its perimeter fence to keep it and its fuel safe) and one of the University's data centres, together costing millions of dollars, and our few hundred dollars worth of resilience mobile communications gear lost in the vista.

Of course, the University data centre does much more than what our little buckets of radio can do, but sometimes you only need a little, and can't afford or sustain the big alternative.

This is some ideas that I plan to implement that I wanted to capture somewhere public before I forget. I plan to implement this in the RFD900 firmware for the Mesh Extenders.

---

The ISM915 band requires frequency hopping in most countries where it is proclaimed.

This is a bit of a pain, because the frequency hopping means that we need to synchronise all the nodes in a mesh such that they are all listening and talking on the same frequency at the same time. That is, they need to be on the same "virtual channel".

The virtual channel is really just a hopping pattern, and knowing your position in the pattern.

It occurred to me some time ago that this imposition actually creates some interesting opportunities that we might not have otherwise considered for cooperative shared use of a block of spectrum.

The first step is to use a fairly fast frequency hopping pattern, hopping perhaps every 1ms - 2ms.

This hopping time is purposely shorter than the time it takes to send a whole packet.

The second step is to have each station only listen on a fraction of slots. For example:

Define a bundle to be a fixed number of, say, 10 consecutive slots.
Each station listens on exactly one of those slots, based, for example, on the last three bits of its' network address.
The remaining two slots are a double-sized slot for broadcast packets. All stations should listen during this slot.
This means that while the logical hopping rate might be 1000Hz, the effective rate for each station is reduced to two hops per slot bundle, i.e., twice every 100Hz, i.e., 200Hz.

This reduces the number of hopping actions for each station, and to some extent allows for a little slop in the synchronisation.

When a station sends a packet to another station, it can predict which time slot to send in, because the receiving stations listen slots can be determined from its network address.

The sender then switches to the appropriate channel at the appropriate time, and sends the entire packet, even though it might take many slots to do so.

In effect, the transmitter and receiver switch from the shared virtual channel to another channel.

This means that each packet transfer "forks" off from the main channel, hence my calling it "multi-threaded radio".

This means that as many packets can be in flight as there are real channels in the hopping pattern, and that they can all be used at the same time, but not by the same stations. That is, it has an intrinsic fair-share mechanism built in.

In terms of aggregate bandwidth, the result is potentially very pleasing.

A 50 channel 128kbit hopping scheme such as we use on the RFD900 radio goes from being 128kbit/second throughput to potentially 50x128kbit = 6.4Mbit/second!

Yet, without the range reduction and increased receiver power consumption that Wi-Fi suffers by trying to make all that bandwidth available to a single station.

Anyway, I need to implement it to see how it works in reality, and whether the accurate synchronisation is possible on a completely distributed network with no external clock.

We have known from the beginning that expecting people to root their Android phones to use the mesh was not the ideal solution. This is why we have put effort into liaising with the IEEE 802.11 committee, as well as encouraging Google to fix bug 82.

Bug 82 is the bug that describes the lack of ad-hoc Wi-Fi support in Android, and was filed back in January 2008, and at the time of writing is the fourth most starred bug in the Android bug tracker. You can see it and vote for it here. To vote for it, just click on the star next to the issue.

While we hope that this will be fixed one day, and ad-hoc Wi-Fi will be easily available on all Android phones, we have put considerable effort into a work-around.

That work-around is to alternate Wi-Fi on a phone between Access Point and Client modes. Combined with the Serval Rhizome store-and-forward protocol, this allows data to move from phone to phone. The downside is that you can't have an end-to-end connection over many hops. Nonetheless, we have created text messaging and file sharing applications, amongst others, over Rhizome and using this step-by-step approach to moving data by Wi-Fi.

More recently, we have sought to mature this ability to operate without root access so that the Serval Mesh software assumes that there is no root access, and to operate as best it can in that situation. This functionality is now more or less complete, and is expected to appear in Serval Mesh 0.91 in the next while. You can try it out by downloading and building the development branch of the Serval Mesh at http://github.com/servalproject/batphone.

To give an idea of how this works in practice, I made a few screenshots of a nightly build of Serval Mesh 0.91-pre:

First, here is the new home screen. The main difference is that the "Switch on/Switch off" button is now called "Connect". This name might change before 0.91 is released.

Instead of just starting or stopping the mesh, it now takes you to a list of available networks:

The operation of the mesh software on the phone itself is now controlled by ticking or unticking "Enable Services". Again, the label may change and indeed we expect to make this whole display much more visually appealing.

The Wi-Fi networks that are available or might be available are listed below. Selecting "Auto Cycle" is still an option to get the mostly-automatic behaviour of previous versions. But it is now much easier to pick a particular Wi-Fi network.

Note also that ad-hoc shows up, even though this is on an unrooted phone. However, it won't be used without user intervention.

To attempt to use ad-hoc mode you touch the "Adhoc" item. It then asks if you would like to try to setup ad-hoc mode, and reminds you that this requires root access, and might not be supported on all phones.

If you respond with "Test", then it tries to get root access and enable ad-hoc Wi-Fi:

In the case of this phone, it fails of course, because it is not rooted. But I can of course choose to join another Wi-Fi network, or just enable Auto Cycle mode, and any collection of phones running the Serval Mesh should then connect to one another or other networks when available:

So while there is some refinement to go, the Serval Mesh software is now able to work easily without root access, and does not try to get root access without asking the user first.

For convenience, here are direct links to four of the five talks we gave at LCA2013 in Canberra:

http://mirror.linux.org.au/linux.conf.au/2013/mp4/Making_Mobile_Communications_Secure.mp4
http://mirror.linux.org.au/linux.conf.au/2013/mp4/Serval_Maps.mp4
http://mirror.linux.org.au/linux.conf.au/2013/mp4/Serval_Project_Technology_Stack.mp4
http://mirror.linux.org.au/linux.conf.au/2013/mp4/Field_Trialling_the_Serval_Project_with_New_Zealand_Red_Cross.mp4

We can't find what happened to the other one, oh well.

This is just a quick post to share a problem I had while trying to update the firmware on an MK808B recently, and the solution I came up with. The problem was trivial, but none of the suggested solutions that I found online were appropriate, so I am sharing what happened here to save other people the same issue.

I downloaded the USB drivers for a device, connected the device, and waited for Windows to ask about drivers for it. I had the drivers downloaded and extracted ready for installation.

When I selected the folder with the drivers, Windows proceeded to try to install the drivers, but then very quickly reported "Access is denied".

The same happened if I tried to update driver from in Device Manager.

The problem was caused by the folder containing the drivers being read-only, and I think, encrypted (but I am a bit fuzzy on what the Encrypted attribute really does on Windows 7).

I solved the problem by right-clicking on the folder, and unticking "read-only" and "encrypted" and applying that to the entire folder and its contents. This took a few seconds for Windows to fix the attributes on all the files.

After that I was able to install the driver without incident.

Hopefully this helps save someone else some hassle.

I am trying to design the 2nd generation Mesh Extender prototypes. These will still be prototypes, and budgetary and time constraints mean that we will need to use off-the-shelf hardware for the most part.

Two of the goals are to reduce the size from bucket sized to book-sized, and to weigh less than a kilo-gram, while still preserving at least 24 hours of run-time. A third goal is to use a much faster CPU so that verifying signatures on Rhizome bundles no longer takes tens of milliseconds.

This introduces a tension on the battery size. The 120WH batteries we have been using would be great, if only they were smaller and lighter. We could of course go for a very small and light battery, but then the Mesh Extender wouldn't be able to run for 24 hours between recharges.

To know just how small we can make the battery, we need to know what our power budget is, and this is what I set about determining today. You can see the test rig in the picture. Basically I measured the current consumed by the USB lead that I was using to power everything, and varied what I plugged in the end.

The display on the TV is the HDMI output of the MK808B which is plugged in out of view behind the TV. The MK802ii is visible just to the left of the Mac, and one of the RFD900 radios connected to an FTDI USB2serial adapter is currently plugged into the USB lead that is powered by the desk supply on the left, and is consuming 0.091A @ 4.995V DC, which implies power consumption of around 0.46W at that particular instant.

I used the TP-LINK WR703N based Mesh Extender prototypes as a base-line, and compared that against MK802ii and MK808B Android "Stick PCs". I also separately measured the power consumption of the USB to serial adapters, and tried to get an idea of the power consumption of the RFD900 radios as well. The raw measurements I made are available here.

The summary of what I discovered is:

Device	Feature	Energy Consumption
TP-LINK WR703N	Base power consumption	0.50W - 0.55W
	Wi-Fi (Access Point Mode + Ad-Hoc Mode) + USB hub + memory stick + running servald	0.46W - 0.50W
	Base + Wi-Fi + servald	0.96W - 1.05W
MK802ii	Base + Wi-Fi Client Mode + HDMI display	1.05W - 1.5W
	Base + Wi-Fi Client Mode	1.0W - 1.50W
MK808B	Base power consumption	0.51W - 0.52W
	HDMI Display Connected	0.21W - 0.22W
	Wi-Fi (Client Mode)	0.21W - 0.25W*
	Wi-Fi (Access Point Mode)	0.21W - 0.27W*
	Base + Wi-Fi	0.73W - 0.79W
CP210x USB2Serial		0.13W - 0.14W
FTDI USB2Serial		0.11W
RFD900	TX power set to 250mW	0.09W - >0.49W*

Entries marked with an asterisk probably underestimate the peak power consumption, because the bench top supply I was using did not have (or probably I didn't know how to use) hold peak current mode that would let me see what the peak current consumption was. Instead, I sat and watched the display and tried to read the lowest and highest numbers off to provide estimated power consumption ranges.

This was mostly a problem with the RFD900 radios where RX and TX power consumption differ significantly, compounded by the radio switching rapidly between the modes.

The results were a pleasant surprise over all, because I had been led to believe that the MK808B used somewhat more than 1W when idle, so to find that it used only about three-quarters of a watt was a pleasing. A significant contributor of this efficiency seems to come from the ability of the MK8080B ROM that I used to disable the HDMI output, saving a fifth of a watt. This doesn't seem to be part of the ROM on the MK802ii that I have here.

When booting or doing heavy CPU load the power consumption of all three devices was substantially higher, as these embedded platform all use a variety of tricks to minimise power consumption.

I was surprised to discover that the MK808B might in fact use less energy than the WR703N, provided that running servald on it doesn't use more than a quarter of a watt. The cores in the MK808B apparently throttle down to 265MHz when there is nothing to do, which should be enough to run servald, so it might be that the energy consumption increases only very slightly under typical loads.

I started to look into measuring servald's power consumption on the MK808B, but ran out of time when I hit an odd problem with the serial connect to the radio on the MK808B, perhaps related to the CP210x adapter. I will revisit this in the next week or so.

So provided the servald energy consumption comes in okay, it looks like the MK808B is a good contender for the next model of Mesh Extender prototype. The main caveat is the lack of simultaneous access point and ad-hoc Wi-Fi, which means that the Mesh Extenders will only be able to talk to each other via packet radio, instead of being able to use Wi-Fi when in close proximity.

All up, it looks like our continuous energy budget, averaging out RFD900 RX and TX consumption @ 250mW (+20dBm) will probably be around 1.3W or so. Allowing for an 80% efficient supply from battery, this means we need 1.65W * 24 hours = 40Wh battery capacity. Allow an extra 10Wh or so for charging a smart phone, and we need a battery of somewhere around 50Wh to meet our needs.

So even though there are a few uncertainties around actual averaged RFD900 power consumption and the power consumption that running servald will introduce, we have a ball-park battery size, and one which is much smaller than the 120Wh monsters we have been using to date. I have ordered a 50Wh battery pack from dealextreme so that I can test the run-time in the next week or two.

Yesterday we finally got around to performing an urban test of the Mesh Extender prototypes.

To make this a realistic test, we have one Mesh Extender just sitting inside my house on a cupboard. There was no antenna aiming. It was not up high. It was inside a double-brick house, placed no better than you could expect an uninformed user to place it.

We know that in similar circumstances that Wi-Fi has a range of about one house, i.e., you can often see your immediate neighbours Wi-Fi, but not any further.

We wanted to see how much further we could cover with the Mesh Extender's 915MHz packet radio.

The first part of the test consisted of walking around the block where we live. This block has 11 houses in a row along its length, with us on one end (the green marker on the map). From where we placed the Extender to the far corner on the footpath is about 192 metres as the photon flies. Transmit power was +24dBm (250mW) and air bit rate was 128kbit.

We were able to maintain link all the way around the block, without it dropping even once. This is very encouraging, as it suggests that anyone on our block would be able to put a Mesh Extender in their house, and we would have digital communications between them. Again, contrast this with Wi-Fi where if you are lucky you can see your next-door neighbours Wi-Fi.

Encouraged by the success of this, we decided to see if we could get a link from my house to the local super-market. The Mesh Extender remained where it was, and we walked around to the super-market. The idea was to push as far as we could until link was lost.

We took the route along Minchinbury Terrace, and then across to the bottom-right corner of the shopping complex, and then walked towards the red mark on the map. The link became intermittent once we were on Chambers Street, and was only reestablished intermittently at the shopping centre. The maximum distance where we had link, in the foyer of Coles, was about 262m.

Remember that these units were only metre or so off the ground, and one was inside a double-walled brick house, and that there were a number of similarly constructed dwellings and large trees in the way. This is really quite a hostile environment, and is certainly not unreasonably optimistic compared with expected real-world deployment in urban areas.

As mentioned above, Wi-Fi range of about 1 house means that forming a suburban Wi-Fi mesh requires almost 100% of homes to participate. In contrast, if we draw a circle around the 192m range where we had solid link, and the 192m-262m range where a link might be possible, the story is very different:

The solid range area includes about 90 separate homes, and that ignores that a few of those are blocks of units. This means that an adoption rate of 2/90 = about 2% would be sufficient to almost guarantee the formation of a continuous mesh, with people just placing the Mesh Extenders in a convenient location inside their home.

If we assume that half the homes in the 192m - 262m range can establish a link, then that adds another 25 houses (not even counting the ones that didn't fit in the image). In fact, in most realistic urban or suburban deployments then dwellings are likely to be packed more closely.

But most importantly, if even a few Mesh Extenders are placed in better vantage points, perhaps on a roof, or even on the top of a book case, then it is possible that the range will be extended even further. Radio range is typically proportional to the square root of the height of the transmitter above ground level, so doubling the height of the radio from ~1m to ~2m could increase the range by about 71%, which would double the number of houses in range. Double the height at both ends, and the effect is combined, and the range would be doubled.

Placing units 4m or so off the ground, e.g., on the roof of a house would likely extend the range much more, as in addition to the 2x range due to increased height, the obstruction of the house and other objects near ground level would be greatly reduced, and distances in the kilo-metres should be achievable in favourable conditions. It seems quite feasible that well placed units could reach potentially thousands to tens of thousands of other homes. This is something that we may try to test in the next few days.

This is the kind of capability that makes mobile mesh telephony able to cover very large areas, and connect whole communities, and without requiring wide-spread adoption to begin forming the network.

And I again emphasise that this is without any aiming or special skills on the part of the people "installing" the Mesh Extenders.

To achieve this we have some hardware design work to do, as well as some interesting work on improving the packet radio firmware. If anyone would like to pitch in and help, we'd love to hear about it.

Following the previous test where we placed a mesh extender on a bench in my house, we decided to test again with the mesh extender located in a better, higher location to simulate a purposeful installation.

The first step was to mod our existing prototypes to have the antennae sticking out of the top of their tubs. This reflects our view that the antenna need to stick out to obtain the best performance. They can be seen here ready to be tested:

One was then promptly installed on our roof:

One thing you will notice is that we have lots of trees in our area that are higher than where we put the Mesh Extender. So we would continue to suffer from a lack of line-of-sight. Nonetheless, we were confident that we would get substantially better range than in the previous test, but only had a short time window to test in. This was a bit of an issue, because we would need to cover a few kilo-metres of roads in about twenty minutes.

We thought about driving around, and getting out periodically to measure signal strength, but that didn't sound like fun, and wouldn't give a good idea of real "on the street" coverage, since cars are pretty good shields. So while it is a situation we should examine, it wasn't the goal of today's test.

Fortunately, I own a Dutch Bakfiets (cargo bike), and one of our developers, was willing to sit in the bike while we rode around. This worked really well, as we could easily cruise around at about 10km/hour - 15km/hour, and Andrew could read signal strengths off as we went around. You can see him in the bike here, holding his phone to view the signal information, and with the Mesh Extender sitting between his feet.

So how well did it work?

Well, first up, I tested performance around at the local super-market, which we could almost but not quite reach with the Mesh Extender inside my house. But this time it was possible to get signal about 10m or so into the super-market building. Here is the Mesh Extender in the fruit-and-veg department showing a solid link of about 10dB:

Then it was time to ride around the neighbourhood and see roughly how far we could get. Last time we had reliable links to around 200m, and some links to about 260m. This time we had reliable links to around 500m, spanning about three blocks:

Several hundred dwellings would be within range of this unit. Again, this is the per-hop range, and once we improve the radio firmware to mesh, it will be quite feasible to cover significant distances with multiple hops.

Part of the rush in this test was that I had to collect Caleb from Childcare. I left the Extender running on the roof, and took the other with me, and came back a different route to see whether we could pick up any signal along Marion Road (the main road running top to bottom in the image). We did indeed pickup a link along portions of Marion Road at a distance of up to about 850m, including the point shown here:

Not entirely surprisingly the link could be picked up on the further side of Marion Road, but not the nearer side, presumably due to RF shadowing.

The Mesh Extender in the Bakfiets was less than 1m off the ground, and raising it higher would most likely have improved things, and suggests that distances of a few kilometres might be possible roof-to-roof with good line of sight.

So overall, the range was at least 2.5x better than with the indoor placement. This is not surprising. It was, however, a pleasant surprise to see urban distances of almost 1km possible -- despite not using advanced error correction and detection schemes in the radio.

Such a roof-top deployment has the potential to provide coverage over hundreds of dwellings, and it is easy to envisage even a relatively modest mesh of perhaps a dozen or so nodes being able cover an entire suburb, and allow people to make use of mesh services by carrying a small Mesh Extender with them to connect.

Today we did some more testing with the prototype Mesh Extenders, and finally got around to filming the Mesh Extenders actually relaying mesh data, rather than just reporting link quality figures.

As a side effect, we also ended up testing some of Jeremy's new mesh routing code.

The topology we used was two un-rooted Android phones and two Mesh Extenders. Each phone was connected to one of the Mesh Extenders as a Wi-Fi client. The Mesh Extenders used their Wi-Fi and UHF packet radios to connect to each other. Thus there were three hops from one phone to the other, as shown in the peer list screen grab below:

Jeremy added the nice routing information to the peer list recently, so that you can see the number of hops to get to a peer, including the route taken. This makes it much easier for us to debug multi-hop paths.

We then put one of the Mesh Extenders on a bench by a window in the lab, and left Jeremy next to it with one of the phones:

I took the other Mesh Extender walk-about and in the RV Bakfiets to see how far we could go, sending and receiving text messages and making the phones ring at each location. An actual voice call over the limited bandwidth of the Mesh Extenders will not be possible until we add support for CODEC2 into batphone. What was encouraging was we did hear pops of audio when we placed calls, indicating that it was only the bandwidth starvation that was stopping us from having voice calls.

The idea was to test reachability, rather than probe the exact limits of range. We are planning a little phone app that will log signal strength and GPS location to help map the limits of range more exactly, but that will be another day. A couple of locations were of the menu today:

First, in the bush behind the University. There the challenges are vegetation and undulating land. The near edge of the bush land where I went was visible through the lab window where the Mesh Extender was sitting, with the building not getting in the way. Here I am in the bush with the Mest Extender and phone after doing the test there:

Here is a map showing approximately where we were. The lab is the green spot, and we were about where the red spot is. We had about +13dB link margin, which is ample.

Here you can see me send and receive text messages, asking Jeremy to call my phone, which he does:

Total distance was about 315m, which was pretty nice, given all the trees in the way, and the there are some low rises in the path as well. Also, the car park was full of cars, so the path wasn't as clear as the satellite image suggests.

Then later in the afternoon I needed to head down to the plaza in the central part of the campus. Discussion in the lab was not very positive about getting to the plaza in one hop.

I rode my bakfiets down, with the Mesh Extender in the box. The first view is roughly in the direction of the lab -- through a pile of trees, and up the hill, and through the entire Engineering building -- there was no good line of sight for this one, just a strong elevated position.

This is the view across the plaza, with a number of people enjoying the unseasonably warm weather this week (28c just three weeks out from winter):

Here I call Jeremy's phone and receive a message from Jeremy while on the plaza:

The distance was about 415m, with +12dB link margin. Riding around the plaza the signal was fine unless I ventured too far East (right on the map) towards the buildings, in which case the obstructions were just too much.

As with the other recent tests of the Mesh Extenders, we continue to see that the distance between each hop can be something like 10x that of Wi-Fi, and hence cover 100x the area, whether in urban areas, open country, vegetated areas or otherwise. The following list of posts is a summary of some of the tests we have performed so far, all lending support to this premise:

http://servalpaul.blogspot.com/2013/05/urban-testing-of-mesh-extender-part-1.html
http://servalpaul.blogspot.com/2013/05/urban-testing-of-mesh-extender-part-2.html
http://servalpaul.blogspot.com/2013/03/testing-serval-mesh-helper-prototypes.html
http://servalpaul.blogspot.com/2013/02/3km-mesh-link-using-mesh-helper.html
http://servalpaul.blogspot.com/2013/02/mesh-link-using-our-prototype-mesh.html

This is quite important, because the human voice has roughly the same propagation characteristics as Wi-Fi, that is, you can typically shout at someone if they are in Wi-Fi range, which means that prior to the Mesh Extender, single-hop mesh communications were basically no further than you could shout at someone.

But now are consistently facilitating communications an order of magnitude further, to a distance that is more like that of a hand-held CB radio in a single hop. But unlike a CB radio, we have rich digital services, and once we complete the firmware rewrite we will be able to carry services over many Mesh Extender hops, thus also eclipsing the range of ordinary CB radio.

So perhaps we should be marketing Serval as something like "Digital CB" rather than just as "mobile mesh telephony"...

I have a DROBO which I am trying to recover with DiskWarrior.

When I connect the DROBO and try to run DiskWarrior, I get an error -36 "hardware failure".

I can hear the DROBO doing "stuff".

If I wait several hours until the DROBO is quiet, then DiskWarrior doesn't get the error, and I can try to recover it.

Scratched my head over this for a while, so figured I would post the solution in case anyone has the same problem.

All the forum posts are about dead disks.

My disk isn't dead, it just has a sick filesystem, as proven by the fact that if I wait long enough, DiskWarrior can try to do something with it.

The disk activity was a clue.

I suspected that TimeMachine or something else on the mac is trying to fsck the file system in preparation for mounting. Unfortunately that takes HOURS on this large volume full of time machine backups.

So opened Terminal and typed: ps -ef | grep fsck_hfs and there was the culprit:

0 67332 18 0 6:26am ?? 1:02.06 /System/Library/Filesystems/hfs.fs/Contents/Resources/../../../../../../sbin/fsck_hfs -y /dev/disk5s2

To kill it, type kill followed by the second number from the left, in this case: kill 67332

The disk activity stops, and you can then run DiskWarrior.

Serval Mesh 0.90 Release Candidate 1 is Out

Serval Technology Stack (Part 2) - Rhizome

Serval Mesh 0.90 RC1 Demo

Breaking the WiFi Barrier - The Serval Mesh Helper Packet-Radio Prototyping (Part 1)

Securing Mobile Telecommunications Talk from LinuxConfAU 2013

Building Serval Mesh Helper Device Prototypes

Mesh link using our prototype Mesh Helper through ~200m of forest

3KM Mesh link using Mesh Helper Prototypes

Testing Serval Mesh Helper Prototypes in Moores Valley

Interfacing between the Serval Mesh and Cellular Networks (Part 1)

Serval versus Infrastructure

"Multi-threaded radio" for the ISM915 band for fun and profit

Root-free operation of the Serval Mesh

Talks from Linux Conf AU 2013

"Access is Denied" when attempting to install drivers on Windows 7

Comparing energy consumption of candidate Mesh Extender components

Urban testing of Mesh Extender - Part 1

Urban testing of Mesh Extender - Part 2

Multi-Hop MeshMS (Mesh SMS) and Ringing Phones via Mesh Extender

DiskWarrior Error (-36 2747) "hardware failure" unless you wait for hours: solution