UNOLS SWAP MEET – San Diego

Results, Decisions and Recommendations

14 January 2003

 

Toby Martin (Oregon State University) 

Val Schmidt (LDEO-Columbia)

Geoff Davis (Scripps/UCSD)

 

Purpose:

            In an ongoing effort to investigate the application of wireless technologies to the UNOLS fleet and port facilities, a collaborative event was held at Scripps Institute of Oceanography.  This was the first step in a series of proposed meetings to provide hardware and operating guidelines for UNOLS vessels for wireless networking interoperability between UNOLS ships, between a ship and a UNOLS port facility, or between UNOLS ships and instrumented buoys or other devices. 

 

The purpose and goals of the UNOLS swap group is summarized in the Story/Scenarios below.

 

Our Story/Scenarios

 

  1. A ship pulls into its home port, and before the ship hits the pier, a wireless network link is made automatically to the local shore network and larger Internet.
  2. Same as 1 except the port is not the ship’s home port, but UNOLS facility offering this service.
  3. A wireless network link is generated automatically between two ships at sea when they are in reception range.  This requires no user interaction, and automated processes that look for the presence of the other ship will execute over the link to dedicated known hosts on the peer ship.
  4. A link is generated automatically between three or more ships at sea when in reception range.  If ship A is in range of ship B but not ship C, ship B can still forward packets on to ship C from ship A.  (“Hidden nodes” are still routable.)
  5. In the case of 3 or 4, if one ship is outfitted with a broadband satellite link to shore (e.g. RoadNet), shipboard personnel can provide the other ships access to that link via the wireless ship-to-ship service. The satellite linked ship maintains usage statistics for link cost sharing between vessels and PIs.
  6. An instrumented buoy outfitted with a wireless radio is externally triggered to attempt a wireless connection with a passing ship.  When the connection is made, the buoy’s data and operating status are downloaded to a known server on that ship.
  7. A ship passing within radio range of a participating shore facility maintains a constant data link to shore.  The link might serve ships operating near institutions without deep water ports (e.g. an Ice Breaker passing Barrow), or ships conducting local coastal surveying, testing instrumentation or whatever.

 

After much investigation we have decided on the following candidate topology, hardware, software configuration and administration.

 

Candidate Topology:

 

General Topological Description:

 

First let us describe some technical details regarding IP networks.  In traditional wired IP infrastructure, packets are transferred between networks or subnets, by routers. Routers use “network addresses” to make routing decisions.  Network addresses comprise the “masked” portion of the IP address. [Hence an IP address of 192.168.1.1 and a subnet mask of 255.255.255.0, the network address is 192.168.1.] A router will use just these bits to decide how to send the packet to its final network.  The routing function within an IP network is a “Layer 3” function within the standard OSI model.  Within a network, packets are passed between physical segments by switches.  Hardware addresses (MAC addresses) and the “host” portion of the IP address (the final 1 in the example above) are used to pass traffic to the appropriate segment and host.  The switching function within an IP network is a “Layer 2” function within the standard OSI model.

 

The 802.11b/g wireless standard specifies two general working models for links between wireless devices.  The first is “ad-hoc mode” which allows two devices to create a single point-to-point connection between them.  They cannot share this connection with other wireless devices, indeed, the wired analog of this mode is a single Ethernet cable strung between the two clients.  The second is “infrastructure” mode, in which multiple wireless clients associate with a single Access Point (AP).  In this mode, switching of traffic between wireless clients is performed by the Access Point and no traffic flows directly between wireless clients.  The wired analog of this mode is a star topology with a switch in the center and single Ethernet segments radiating out to each client. Note that the AP does not “route” traffic between clients.  All clients and the AP must be on the same IP network, hence the AP acts as a switch as described above.

 

Neither of these standard modes will meet the needs of the UNOLS fleet.  Ad-hoc point to point connections between single clients do not inherently allow for the routing of traffic either between daisy chained hosts or to shipboard or shore wired networks beyond the wireless client.  Infrastructure networks require that a single device be preconfigured as an Access Point and none others – a provision for which we cannot provide without user intervention for each attempted link.  Moreover, should the AP loose sight of a client device, no traffic will flow from that device even though other client devices may be near by.

 

Therefore we have searched far and wide to find combination router-wireless devices that will meet the requirements we have set. At the moment, we have been able to find no commercial implementations that have the requisite functionality. However, we have conducted preliminary tests on a set of open source tools that we believe will provide all the functionality we require, as well as provide for easy implementation of new functionality in the future. We have come to call these router-wireless devices “SWAP devices.”

 

SWAP Devices

 

The hardware for this device is a single board PC.  At a minimum these have two RJ-45 Ethernet ports, a serial-console port, two PCMCIA slots and a compact flash slot.  Wireless PCMCIA radio cards are purchased separately and installed to provide wireless functionality. This is desired to allow for easy upgrades in radio technology (for example to 802.11g).  The compact flash card holds the operating system, making for easy upgrades with preconfigured, test flash card replacements.  Of course, the system will be installed in a weather-proof enclosure and will include wireless antennas appropriate to the ship.

 

At the core of this device is the Pebble Linux operating system – a derivative of Debian Linux designed expressly for Linux based wireless devices.  Several ancillary software packages provide crucial functionality – HostAP creates associations with other wireless devices, Zebra provides dynamic routing protocols (particularly OSPF), Aladin creates dynamic pseudo network interfaces when wireless associations are made between two SWAP devices. 

 

SWAP-Peer Associations

 

When any two SWAP devices come into reception range, a wireless ad-hoc link is established between them automatically. However unlike typical wireless associations, the ad-hoc link is established on top of a dynamically created pair of pseudo wireless interfaces whose IP addresses are taken from a pre-established pool.  The wireless space between the devices is now a unique, routable subnet, rather than an extension of a wired subnet as the 802.11 standard would otherwise provide.  We call this a “SWAP-Peer” association. [This special association only happens when both devices are SWAP wireless/router devices configured with the same ESSID described above.]

 

To pass traffic between the wireless “swap space” and wired networks on either end, the two wireless-router SWAP Devices must exchange routing tables.  To facilitate this each operates the OSPF (Open Shortest Pass First) dynamic routing protocol which provides for automated exchange of routing tables between routers. 

 

When a third SWAP device comes within range of the other two, wireless ad-hoc links are established between each of them, again atop pairs of pseudo wireless interfaces creating unique subnets. Again, when the SWAP-Peer associations are created, the local routing table is updated and OSPF broadcasts the new route to the other local routers. In this way, the routing table of each SWAP device is updated to reflect the new links.   

 

When a SWAP device drops out of range of the one of the other two, its wireless link and pseudo wireless interfaces are destroyed.  In turn, the local routing table is updated to reflect the loss of this link, and changes to the routing table are broadcast to the SWAP device at the other end of the remaining link.  These changes are propagated to the third SWAP device, such that traffic can still be passed between all nodes, now via the central SWAP device.

 

Interactions between ships and shore facilities are essentially identical.  Shore facilities install identical SWAP devices to those on ships at sea.  SWAP-Peer associations and the propagation of routing tables occur in an identical way. 

 

In our discussion thus far, we’ve left out some details that might best be illustrated with an example

 

An Example:

Consider the diagram above, depicting three ships at sea and a Port Facility.  First let us discuss Ship A and its network topology. 

 

Ship-board Networks 

In the SHIP A box above, the white square depicts the ship’s internal network.  We have given it a 192.168.0.0/16 subnet for the sake of discussion, but in actuality, many ships use public network spaces, often with multiple subnets.  The details of the ship’s internal network are largely immaterial to this discussion, other than its connection to the SWAP network, more on that in a moment. 

 

The green square depicts the ship’s local  SWAP network.  This network is distinguished from the dynamically created networks that are created between SWAP devices described earlier. Rather, it is a network local to the ship that is completely routable from the SWAP space.  We have set a few guidelines regarding this network, namely, the first two numbers of the SWAP network address will be the same for all SWAP installations.  The third number will be unique to each ship or shore facility.  So for Ship A, the SWAP network is 10.20.101.0/24 (We have initially chosen Class C subnets for the SWAP network, which will allow us to support 254 unique SWAP networks (and hence 254 unique SWAP Devices) each with 254 possible nodes, without a large revision of our numbering scheme.  We may revise this in the future.) Within the local SWAP Network, we have mandated a few host IP addresses.  The SWAP device itself will always reside at 10.20.XXX.1.  Should the ship chose to host a SWAP server, it will always reside at 10.20.XXX.2.  We reserve the right to set additional policies regarding additional mandated IP addresses in the SWAP space.  The need for this will become apparent momentarily.

 

The SWAP Device

The upper right hand corner of the Ship A diagram depicts the composite SWAP Device – a Radio Antenna, router and Network Address Translation Device.  The SWAP Device has three static network interfaces (distinguished from the dynamically created WDS interfaces).  Eth0 is connected to the SWAP network, eth1 is connected to the ship’s internal network, and wlan0 is the wireless radio.  (wlan0 is not shown.) 

 

Obviously, the addition of the SWAP infrastructure must keep the current ship’s network intact. Moreover, we did not want to populate either the ship’s routers or the SWAP routers with routing information regarding the other’s network. Therefore, we have buffered the connection between the ship’s internal network and the SWAP networks with a NAT. Network Address Translation (NAT) provides the translation of multiple IP addresses on one network to that of a single IP address on a second network, all the while keeping track of which packets are for which host.  Typically NAT requires that initial network connections be made from translated hosts, as there is no way to identify which translated host an initial connection is intended for.  Hence the NAT allows hosts on the ship’s internal network to connect to hosts in the SWAP infrastructure, but those in the SWAP networks will not be able to initiate connections to the ship’s internal network hosts.

 

We mentioned parenthetically above that the wireless static interface wlan0 is not shown in our diagram.  There is good reason for this and we thought we’d better discuss it.  We will not actually configure the wlan0 interface with an IP address on any of the SWAP devices.  If we do, the HostAP software will allow client-AP connections to the SWAP devices.  “Great!” you say.  “The SWAP Device can could both create Point-to-Point connections between ships, and also provide wireless AP service to clients local ship – say a laptop on deck.”   “But there’s a hitch!” we say.  It turns out, if a second ship or shore facility is in close proximity, a wireless client device (say a laptop out on deck) will randomly select either SWAP device with which to associate.  (Remember, they both have the same ESSID.)  So for example, you go out on deck with your laptop and it associates with an AP with the SWAP ESSID.  You configure your TCP/IP settings for a gateway IP address of what you think wlan0 on your local swap device is configured.  But you can’t pass any traffic, and you’re dumbfounded for several hours, until you finally realize that you’ve associated with the other ship.  Now what?  You can try to disable and re-enable you’re wireless card to force a re-association in the hopes that you get picked up by your local AP.  Or you can contact the distant ship and ask for their wlan0 IP address to configure your gateway TCP/IP settings.  But the former is a crap shoot - you could be there forever, while the latter will result in an awful connection as all your local traffic will first be passed to the distant ship, before being routed back over the SWAP-Peer link. None of this was satisfactory to us.  The third, much more reasonable solution is to provide local wireless services via a second set of radios with separate ESSID settings.  And we decided this was best done by individual ships as they see fit.

 

The Local SWAP Network

We mentioned earlier that the NAT prevents connections from another ship or shore facility, via the SWAP space, into the ship’s internal network.  “Well how then, will any connections be made at all?  What’s the point, if there’s nothing to which to connect?”  Glad you asked.

 

Enter the Local SWAP Network.  This network, local to each ship, is where hosts are placed for connections from the other ships, port facilities, buoys or whatever.  At least one SWAP server will exist there, (at 10.20.XXX.2 as mentioned above), however ships may add additional devices and servers as their needs and imagination grow.  Indeed, any scientist can configure a laptop or server they’ve carted along and plug right in. 

 

As an example, consider two ships operating in close proximity in which an automated routine synchronizes the contents of a directory on each ship’s SWAP server.  The SWAP servers might share these directories on the local network (via Windows sharing or Samba) such that non-sophisticated uses can drop files to send to the other ship, and retrieve files that have been sent back. 

 

Additionally, the SWAP servers might host email services allowing transfer of email between ships without passing it to shore. 

 

We leave other ingenious uses to the needs and imagination of the fleet.

 

SWAP–Peer Interactions [SHIP to SHIP]

Ok, now let us rehash the details regarding connections between SWAP Devices between ships or ships and shore facilities. First we’ll discuss how the connections are created, and then we can tackle how packets are routed.

 

Operating on each SWAP Device is the HostAP software.  This open source software initiative provides basic Access Point functionality atop the Linux operating system.  Moreover, it provides for dynamic creation of Wireless Distribution System (WDS) links, which provide point-to-point wireless connections and create pseudo network interfaces.  http://www.trekweb.com/~jasonb/articles/hostap_20030727.shtml)

 

So for example, when two ships with SWAP Devices configured with the UNOS SWAP ESSID (yet to be determined) come within reception range, the HostAP software operating on each device automatically makes a WDS link between them.  In the process each SWAP device creates a pseudo network interface for use with that link.  The first pseudo network interface created for a WDS link on a single SWAP device will be designated wlan0wds0, the second wlan0wds1, etc. 

 

Now however, both wds interfaces on both the local and distant SWAP Devices must be provisioned with unique IP addresses on the same subnet to allow traffic to be routed between them.  A second software package called “Aladin” operating as a daemon on both routers, assigns the IP addresses from a pre-configured pool.  [It is not yet clear to us which of two otherwise identical SWAP Devices assigns the IP addresses and hence which pool they come from, but we cannot see that it matters either way provided pools assigned to each device do not overlap.] http://www.sown.org.uk/index.php/Linking

 

Now we have a wireless link between the two SWAP Devices and a subnet dynamically created as well. But without any kind of routing rules and protocols we won’t be able to pass traffic between the local ship SWAP networks to the WDS link networks.  Therefore a third piece of software provides for the routing of traffic between networks and the passing of routing tables between SWAP Devices.  The software package, “Zebra”, provides a selection of several TCP/IP based routing protocols to dynamically create routing tables from local known networks and to share these routing tables with other peer routers.  We have chosen to implement the OSPF (Open Shortest Path First) routing protocol from the Zebra package for our SWAP devices.  http://www.zebra.org/

 

If you remember, a routing table consists of a list of known networks, the interface on which those networks should be found, and optionally a “via ipaddress” entry when the network is not local.  The “via” entry specifies the IP address of a peer router for the next hop.  Peer routers exchange routing tables such that they may learn how to route packets beyond their local networks. 

 

OSPF works somewhat differently than a simple exchange of routing tables between routers.  Rather, OSPF daemons exchange “Link State Advertisements” which designate the location and weighting of local networks.  Each router then calculates a “shortest path tree” from the complete Link State Database with itself as root.  The shortest path tree, in turn, is used to create the routing table. 

 

This level of complexity is important, as it allows for quick convergence of new network paths when nodes are lost.  It also allows one to designate multiple paths to the same network, with differing weights.  One can envision a scenario in which two links to shore exist, and one would like to specify a preference for the least cost route.

 

Before the WDS link is created between Ship A and Ship B, the local ospfd daemons create local routing tables with just the local networks in it. 

 

SHIP A

 

Network

Destination

10.20.101.0/24

Eth0

 

SHIP B

 

Network

Destination

10.20.102.0/24

Eth0

 

Then when the first WDS Link is made, say for example between Ship A and Ship B in our diagram, ospfd creates a new entries for Ship A’s routing table to specify both the WDS Link network, and the Ship B local SWAP network.  A similar event occurs for Ship B.

 

SHIP A

 

Network

Destination

10.20.101.0/24

Eth0

10.10.101.0/30

Wlan0wds0

10.20.102.0/24

Via 10.10.101.2

 

SHIP B

 

Network

Destination

10.20.102.0/24

Eth0

10.10.101.0/30

Wlan0wds0

10.20.101.0/24

Via 10.10.101.1

 

Now the SWAP networks on Ships A and B are fully routable; each routing table contains entries for local networks (eth0 and wlan0wds0), as well as the peer swap networks (via the peer SWAP device wlan0wds interface).  Moreover, hosts on Ship A’s internal network can initiate connections to Ship B’s SWAP server, and those on Ship B’s internal network can initiate connections to Ship A’s SWAP server.  The internal networks on each ship remain hidden from that of the other.

 

When a third WDS link, say between Ship A and Ship C comes up, ospfd creates two new entries in Ship A’s routing table, again for the WDS Link network, and the Ship B local SWAP network. Routing table entries for Ship C are also propagated to Ship B, as are those for Ship’s A and B back to Ship C.  Here again, all local SWAP networks are fully routable from any of the other ships.

 

SHIP A

 

Network

Destination

10.20.101.0/24

Eth0

10.10.101.0/30

Wlan0wds0

10.10.101.4/30

Wlan0wds1

10.20.102.0/24

Via 10.10.101.2

10.20.102.0/24

Via 10.10.101.6

10.20.103.0/24

Via 10.10.101.6

10.20.103.0/24

Via 10.10.101.2

10.10.102.0/30

Via 10.10.101.2

10.10.102.0/30

Via 10.10.101.6

 

SHIP B

 

Network

Destination

10.20.102.0/24

Eth0

10.10.101.0/30

Wlan0wds0

10.10.102.0/30

Wlan0wds1

10.10.101.4/30

Via 10.10.101.1

10.10.101.4/30

Via 10.10.102.2

10.20.101.0/24

Via 10.10.101.1

10.20.101.0/24

Via 10.10.102.2

10.20.103.0/24

Via 10.10.102.2

10.20.103.0/24

Via 10.10.101.1

 

SHIP C

 

Network

Destination

10.20.103.0/24

Eth0

10.10.101.4/30

Wlan0wds0

10.10.102.0/30

Wlan0wds1

10.10.101.0/30

Via 10.10.102.1

10.10.101.0/30

Via 10.10.101.5

10.20.101.0/24

Via 10.10.101.5

10.20.101.0/24

Via 10.10.102.1

10.20.102.0/24

Via 10.10.102.1

10.20.102.0/24

Via 10.10.101.5

 

Now the routing tables have propagated such that, for example, SHIP A, has entries for both the wds links to the peer SWAP devices, two entries for the two possible routes to the wds link between SHIP B and SHIP C, and two entries each for the two possible routes to each of the peer SWAP networks.  Also included in the table, but not shown here, is a metric that tells the SWAP device which network path is the shortest (Open Shortest Path First). 

 

Now that we’ve described all this, let us consider the user’s perspective on Ship A.  Suppose we want to make a connection the Ship B’s SWAP server from a host on Ship A’s local, internal network.  We simply reconfigure our TCP/IP settings to have a gateway IP address of our local SWAP Device.  Then when we look up the Local SWAP Network address for Ship B – in this case 10.20.102.  We know Ship B’s SWAP server will always have a host address of 2, so our target address is 10.20.102.2.  The rest magic.

 

SWAP–Peer Interactions [SHIP to SHORE]

So far we have only considered Ship-to-Ship interactions, but what about connections to shore?  Shore facility SWAP devices are essentially identical to shipboard installations with one exception.  Because we cannot satisfactorily secure 802.11b wireless networks, SWAP devices should never be connected directly into the institution’s private network (behind the firewall).  Rather it should provide access directly to the public Internet.  This will require network address translation to a public Internet address.  Therefore, the NAT is placed between the shore facility SWAP Device and the public Internet. 

 

“How then are connections made between the ship and shore?” you ask.  By design, connections between the ship and shore must be initiated from the ship.  For a host of security reasons, we do not want the shipboard SWAP networks accessible from the public Internet when wireless connections are up.  However, connections may be made from the ship to the shore facility over the public Internet. VPN or IP-Sec tunnels can provide secure connections over the link, the result of which would be a logical direct link between the ship’s internal network and its host institution.  Because these details are site specific, we leave them to individual institutions.

 

SWAP–Peer Interactions [SHIP to Broadband Link]

One of the scenarios outlined above is to provide the ability for multiple ships at sea to share a single broadband link to shore. [For example, one of the ships is RoadNet equipped.]  The configuration for this looks much like that of a Port Facility. More on this at a later date. 

 

SWAP–Peer Interactions [SHIP to Buoy]

There are two ways we envision this could work and we frankly haven’t made sufficient progress testing the basic functionality of everything else to think this through fully.  But for the sake of not loosing our thoughts we’ll outline them here.

 

First, because 802.11 radios consume too much energy to be operated from most battery powered buoys, an out-of-band trigger will be necessary to power on the SWAP radio.  The power to the radio could be automatically switched off after some pre-set time, or the data transfer process could be sufficiently smart to power down when the data transfer is complete.  Either way, the SWAP equipment would remain powered down most of the time.  The out of band trigger should be something unlikely to be set off by accident.  A device derived from an acoustic release has been considered.

 

One method is to outfit the buoy with an entire SWAP device (radio, router, local SWAP Network). In this way the Ship-to-Buoy interaction would be identical to a Ship-to-Ship interaction.  However, since the Buoy would not likely have a way to know a priori what ship network to route to, it might have to check all possible SWAP device networks to scan for an available SWAP server. 

 

A second option is to enable the Buoy with just a client 802.11b wireless device. This option would require configuration of the Ship SWAP Device as an Access Point (i.e. assigning it’s wlan0 interface an IP address) which we opted NOT to do to prevent inadvertent associations with non-local ships. However, it would not be difficult to provide a web-based configuration page in which a Marine Tech places the SWAP device in Buoy mode, thereby enabling AP functionality for a short time.

 

The buoy will of course need to be provided with sufficient smarts to look for a possible SWAP server [or perhaps some other special-function server outfitted to the ships] and when one is found, conduct a data transfer to the server.  We think it should also transfer a meta-data tag that tells the SWAP server where to send the data store when a connection to shore is available, what kinds of links may be utilized, and depending on the link, some billing mechanism for cost recovery.  Clearly there are myriads of details to be worked out here, but the network infrastructure portion of the project seems quite straight forward.

 

Conclusion

In summary our short two days of discussion and testing in San Diego has reaped what we believe is 1) a general selection of hardware and software with the requite functionality and 2) a general topology and configuration that seems to meet the goals set forth by the Story/Scenarios identified at this years RVTEC meeting. 

 

However there is still much to do and test. Among those are:

 

 

The upcoming collaborative event in Hawaii will attempt to address these and many other details.