- Seafloor Mapping Software:
- Processing and Display
- of Swath Sonar Data
-
- David
W. Caress (1), Dale N. Chayes (2, 3) and
Christian dos Santos Ferreira(4)
- 1. Monterey Bay Aquarium
Research Institute
- 2. Center for Coastal and Ocean Mapping/Joint Hydrographic Center
- of the University of New Hampshire
- 3. Lamont-Doherty
Earth Observatory
- of Columbia University
- 4. MARUM (Zentrum fur Marine Umweltwissenschaften)
- (Center for Marine Environmental Sciences)
- of Universitat Bremen
- Introduction
- MB-System is an open source software package for the processing and
display of bathymetry and backscatter imagery data derived from
multibeam, interferometry, and sidescan sonars. The source code for
MB-System is freely available (for free) by anonymous ftp (including
"point and click" access through these web pages). A complete
description is provided in web pages accessed through links below.
- MB-System was originally developed at the Lamont-Doherty Earth
Observatory of Columbia University (L-DEO) and is now a collaborative
effort between the Monterey Bay Aquarium Research Institute (MBARI) and
L-DEO. The National Science Foundation has provided the primary support
for MB-System development since 1993. The Packard Foundation has
provided significant support through MBARI since 1998. Additional
support has derived from SeaBeam Instruments (1994-1997), NOAA
(2002-2004), and others.
- Current Distribution Release Status
Latest Release: 5.5.2284 (23 October 2016)
Last GMT-4 Compatible Release: 5.4.2220 (27 February 2015)
Recent Announcements
- MB-System 5.5.2284 Now Available (23 October 2016)
- This is now the current release of MB-System.
- This version requires GMT 5.2.1.
- For more information read the MB-System 5.5.2284 ChangeLog
file.
- The MB-System 5.5.2284 source distribution is available for download.
- The MB-System Subversion source code archive can also be accessed directly. The third number in the release identifier indicates the corresponding revision level in the archive. Users can, if desired, export old releases from the archive using this number.
- Fixing MBeditviz problems when editing previously edited data (8 July 2016 by David Caress)
- At MBARI we have experienced instances in which bathymetry editing done in mbeditviz failed to show up in the processed files after mbprocess was run. In all cases this problem involved new editing done on previously edited data.
- There have been recent changes to the edit save files (*.esf suffix) used by mbedit, mbeditviz, mbclean, and mbareaclean to hold bathymetry editing events and applied by mbprocess when producing processed files. In particular, the timestamps used to associated edit events with individual sonar pings (or lidar scans) are now handled with a precision exceeding 1 microsecond. This follows less recent changes to the fast bathymetry files (*.fbt suffix) in which timestamps also started being handled to microsecond precision.
- These changes to increase the precision to which time in handled internally in MB-System derive from some of the current generation acoustic and optical seafloor mapping sensors that can have separate data records with ping or scan time stamps separated by 10 milliseconds or less.
- The problem arises from having timestamps in *.esf files and/or *.fbt files that differ from the original timestamps by more than the amount now used to associate edit events with particular pings or scans (0.11 milliseconds). Logic has been embedded to identify and correct these problems on the fly, but that logic clearly fails in some instances. However, these problems can be corrected using a combination of mbdatalist and mbclean.
- If one finds that interactive edits are not reflected in grids produced after processing with mbprocess, do the following:
- First remake the *.fbt files using mbdatalist using the -N option to force creating new ancillary files:
mbdatalist -I datalist.mb-1 -N -V
- Reset the edit event timestamps in the *.esf files using mbclean:
mbclean -I datalist.mb-1 -T0.0011
- The -T tolerance option sets the maximum time difference between the edit event timestamp and the ping or scan timestamp allowed. Since the maximum time difference that should have been produced by rounding errors in previous versions is about 1 millisecond = 0.001 second, I recommend using a tolerance of 0.0011 seconds (works for me...).
- Finally, reprocess the data using mbprocess:
mbprocess -I datalist.mb-1
- As always, if you follow the convention of naming the primary datalist file "datalist.mb-1", then the "-I datalist.mb-1" arguments can be omitted from the mbdatalist, mbclean, and mbprocess calls. If you use some other name for the datalist file, then you must specify it in the commands.
- MB-System 5.4.2220 is the last release compatible with GMT4 (27 February 2015)
- No further changes or fixes will be made to MB-System 5.4 releases.
- We recommend that users migrate to GMT5.1 (or later) and MB-System 5.5 (or later)
- The MB-System 5.4.2220 source distribution will remain available for download through the end of 2015..
Not So Recent Announcements
- MB-System Workshop at MBARI on March 31, 2016 (15 March 2016)
- The Monterey Bay Marine GIS User Group will be holding its annual meeting at MBARI on Wednesday March 30, 2016.
- In conjunction with this meeting, David Caress will present a one-day MB-System workshop the next day (Thursday March 31).
- The primary focus will be on processing surveys done with submerged platforms such as AUVs and ROVs, including optimizing imperfect subsea navigation so that features match in overlapping and crossing swaths.
- If you plan to attend the MB-System workshop, please RSVP.
- If you plan to attend the Monterey Bay Marine GIS User Group meeting, please register on their site.
- MB-System fbt file format documentation examples improved (23 July 2014)
- Documentation of the file format used for "fast bathymetry" or fbt files in MB-System has been improved with the incorporation of C source code for a working program to read fbt files.
- The updated documentation and source code example can be found here.
- The sample source code is licensed under the Lesser GNU Public license, or LGPL and so can be used in the integration of a third-party, commercial bathymetry editor with MB-System
li>
- This format is also known as MBF_MBLDEOIH, or format 71.
- MB-System Developer's Workshop Held (17 April 2014)
- MB-System Developer's Discussion List (17 April 2014)
- For several years we have used the MB-System Discussion List as the primary means for communication
within the MB-System community.
- We have recently begun using a separate but similar email list for discussions regarding MB-System development, leaving the original list for all other sorts of discussions.
- Effectively, we now have two lists:
- Both lists require membership for posting, and have publically viewable message archives.
- If you want to follow everything that is happening with MB-System, subscribe to both lists. If you don't want to see lengthy threads on coding details, just subscribe to the MB-System User's Discussion List.
- MB-System Packaged for HomeBrew on Mac OS X (16 April 2014)
- MB-System has been packaged for MacOs X as part of Homebrew.
- If you have Homebrew installed, then MB-System and its prerequisites can be installed using these two commands:
- brew tap homebrew/science
- brew install mbsystem
- Thanks to Anthony Lukach for creating and maintaining the Homebrew MB-System package.
- MB-System NSF Proposal Funded (October 7, 2013)
- The Marine Geology and Geophysics Program of the NSF has
approved our request for another five years of funding for MB-System
development and support.
- We thank all of the MB-System users who provided letters of collaboration or support for this proposal. Your time and effort are
appreciated!!
- We also thank our NSF program manager Donna Blackman for shepharding this project through the system at a very difficult time.
- The proposal can be read here
- Our proposal was mostly funded intact. Requested effort to develop and maintain the software are fully funded. The documentation effort was cut by 25%, and funds to support annual workshops were also cut.
- New Release of OSGeo-Live (including MB-System) (26 February 2013)
- OSGeo-Live Version 6.5 was released on February 26, 2013.
- OSGeo-Live is a self-contained bootable DVD, USB thumb drive or Virtual Machine based on Xubuntu, that allows you to try a wide variety of open source geospatial software without installing anything.
- This distribution includes over 50 open source geospatial software packages, including MB-System 5.3.1982.
- Hamish Bowman is responsible for packaging and including MB-System into OSGeo-Live.
- The distribution is found at: http://live.osgeo.org/
- The MB-System quickstart description is at: http://live.osgeo.org/en/overview/mb-system_overview.html
- MB-System Reviewed in LIDAR Magazine (6 January 2013)
- MB-System Proposal Status (January 6, 2013)
- Our program manager at the Marine Geology and Geophysics Program, Ocean Sciences Division, U.S. National Science Foundation has indicated that the MB-System renewal proposal reviewed well and is on the list of projects that are high priority for funding.
- However, no funding decisions will be made until the 2013 and 2014 NSF budgets are settled, a process that may very well take another few months.
- MB-System Proposal Submitted (October 15, 2012)
- On October 15, 2012, we submitted a proposal to the Marine Geology and Geophysics Program, Ocean Sciences Division, U.S. National Science Foundation requesting a five-year renewal of funding for MB-System support and development.
- This was structured as two linked, collaborative proposals from MBARI (Principle Investigator David Caress) and Lamont-Doherty (Principle Investigator Dale Chayes).
- The proposal title is: "Collaborative Research: Development and Support of the MB-System Software Package for Processing and Display of Swath Mapping Sonar Data".
- The proposal text is here: 2012 MB-System NSF Proposal
- Supplementary information supporting the proposal is here: Supplementary Information for the 2012 MB-System NSF Proposal., and includes:
- Collaboration letters.
- MB-System user community size estimate.
- Publications citing MB-System
- Estimate of the MB-System User Community Size (October 15, 2012)
- For a variety of reasons, we need to document the existence of a significant MB-System user community.
- In February we estimated the number of organizations using MB-System during 2011 through a combination of Discussion List membership and source distribution download statistics.
- As part of resubmitting our NSF proposal, we have updated our estimate using current Discussion List membership.
- We currently identify 197 user organizations, including 41 academic institutions in the U.S. and 68 elsewhere in the world, 11 U.S. government agencies and 40 government agencies in other countries, and 37 commercial organizations.
- The details are here: MB-System User Community Estimate from Downloads.
- MB-System fbt file format (and documentation thereof...) changed (16 May 2012)
- As of release 5.3.1955, the file format used for "fast bathymetry" or fbt files in MB-System has changed.
- In particular, the navigation, sonar depth, and related parameters are now stored as binary double rather than scaled binary short integer values, with the result that cm-scale surveys can now be usefully processed with MB-System.
- This format is also known as MBF_MBLDEOIH, or format 71.
- Documentation of both the new and old format, along with directions and source code to enable the integration of a third-party bathymetry editor with MB-System, can be found in this updated document.
- MB-System on Facebook (February 29, 2012)
- MB-System Planning Meeting (February 10, 2012)
- On January 17-18, 2012, we held an MB-System Planning Meeting at Lamont-Doherty Earth Observatory to develop the priorities for our NSF proposal (to be submitted on February 15, 2012.
- This was a small meeting involving the core MB-System development team, users that have substantially contributed to the project in recent years, and representatives of organizations that have expressed interest in parallel development efforts contributing to MB-System
- The goals were to:
- Define priorities and tasks for the next five year MB-System project to be proposed to NSF-OCE in February by Caress and Chayes.
- Define priorities and tasks for MB-System related development efforts by other groups.
- The details are here: Notes of and results from the MB-System planning meeting.
- HYSWEEP HSX Format Support Funded by Ocean Floor Geophysics (10 January 2012)
- Ocean Floor Geophysics of Vancouver, Canada, has funded the development of an MB-System i/o module for multibeam data in the HYSWEEP HSX format.
- This capability is available beginning with the 5.3.1917 release distribution. HYSWEEP HSX has an MBIO format id of 201.
- Also included is a new preprocessing utility for data in the HSX format, named mbhysweeppreprocess.
- At this time, the new code has only been tested with respect to data collected using the combination of an R2Sonic multibeam and an Octans gyro/MRU. There will undoubtably be fixes and future development required to successfully support HSX data generally.
- MB-System, GMT, and netCDF RPM Distributions Available from Scripps (December 1, 2010)
- The Scripps Institution of Oceanography's Shipboard Technical Support group maintains a repository of open source software RPM distributions for Red Hat Enterprise Linux and CentOS Linux.
- This repository includes both source and binary distributions of MB-System, GMT, and netCDF that can be accessed using the yum utility. The MB-System RPMs require the GMT and netCDF RPMs, but have no other external dependencies.
- Thanks to Frank Delahoyde for pointing us at these repositories.
- This repository can be found at:
- The SIO-STS gpg signing key is
- MB-System Code Repository Now Publically Accessible (January 7, 2010)
- The MB-System source code repository is now publically accessible through a web interface.
- Users can download individual files or the entire MB-System development structure.
- This repository always reflects the current state of MB-System development.
- Update to Integrating Third-Party Bathymetry Editors
with MB-System (December 31, 2009)
- This
updated document addresses the issue of integrating a third-party, proprietary source
bathymetry
editor with the GPL-licensed MB-System
processing environment.
- In particular, the edit event definition and related source code have been augmented to handle data from multiple head sonars that produce multiple pings with identical timestamps.
- For the
most part, integration of a bathymetry editor with MB-System can be accomplished using executable programs that are
part of the MB-System
distribution, and therefore freely available to
be installed by users independent of any proprietary software.
- However, we also include
some code derived from the MB-System
distribution that we will
separately release under the Lessor
GNU Public
License (LGPL), allowing those
functions to be linked with proprietary software.
- Directions for Installing GMT and MB-System on Ubuntu (June 14, 2009)
- On April 18, 2009, Marcel Croon posted instructions for installing GMT and MB-System on Ubuntu systems to the MB-System Discussion List.
- A slightly edited version of these instructions are available here.
- Thanks Marcel!
- MB-System Packaged for MacOS X and Poseidon Linux (December 31, 2008)
- This isn't really new, but we haven't previously highlighted these efforts to package MB-System for easy installation.
- Since 2004, Kurt Schwehr of CCOM/JHC has maintained a Fink Project MB-System package for MacOs X. Users must specify use of "unstable" packages to make the MB-System package available.
- Since July 2008, MB-System has been included in Poseidon Linux, the "Scientific GNU/Linux". Poseidon is an Ubuntu Linux based distribution including a selection of open source software packages intended to support scientific research. In particular, it offers several specific tools in the areas of GIS, 3D Visualization, Mathematics, Statistics and several other fields of research.
- MB-System Discussion List has changed (May 16, 2008)
- As of May 11, MBARI changed the list server hardware and software that hosts the MB-System Disccussion List
- The MB-System Discussion List URL has changed to http://listserver.mbari.org/sympa/info/mbsystem
- Anyone can read the list archives. Click on "Archives" in the web interface linked above.
- Only members can post to the list. Join the list via the web interface linked above.
- List members can post via the web interface or by emailing to mbsystem@mbari.org
- MB-System Beta Distributions Now Linked in Web Pages (May 16, 2008)
- Beta Distributions of MB-System are generated much more frequently than "official" releases, mostly because we have difficulty keeping the documentation concurrent with the code development.
- The sections of these web pages relating to downloads now note that beta distributions are frequently updated, and include a link to the most recent beta distribution.
- Version 5.1.1 will be released as soon as reasonable documentation is completed for the new tools mbgrdviz and mbeditviz.
- MB-System NSF Proposal Funded (July 26, 2006)
- The Marine Geology and Geophysics Program of the NSF has
approved our request for another five years of funding for MB-System
development and support.
- We thank all of the MB-System users who provided letters of
support for inclusion in this proposal. Your time and effort are
appreciated!!
- The proposal can be read here
- Update to MB-System Cookbook Now Available
(January 14, 2006)
- We are developing an MB-System Cookbook to provide tutorial
style documentation for MB-System Version 5. An updated
draft of this cookbook by Val Schmidt, Dale Chayes, and David Caress is
being made available here along with the data samples used to
illustrate the use of our software (see
links below).
- First Visualization-Aided MB-System Tool Released
(February 20, 2005)
- MB-System version 6 is already under development. Our intent
is to embed a 3D vizualization capability into tools used for tasks
such as project management, bathymetry editing, and patch tests. The first
tool using this capability is part of the 5.0.6 release. This program is
called MBgrdviz, and allows simple 2D and 3D vizualization of GMT grids. Some
screen dumps are available here.
- Integrating Third-Party Bathymetry Editors
with MB-System (October 6, 2004)
- This
document addresses the issue of integrating a third-party
bathymetry
editor with the MB-System
processing environment.
- Since some editors
may have licensing and distribution schemes (e.g. commercial with
source held proprietary) that are inconsistent with the GNU Public
License (GPL) applied to MB-System,
the suggestions presented here do
not involve direct use of GPL
licensed MB-System
source code.
- For the
most part, this can be accomplished using executable programs that are
part of the MB-System
distribution, and therefore freely available to
be installed (but not sold) by third parties.
- However, we also include
some code derived from the MB-System
distribution that we will
separately release under the Lessor
GNU Public
License (LGPL), allowing those
functions to be linked with commercial software.
- Draft MB-System Cookbook Now Available
(December 5, 2003)
- We are developing an MB-System Cookbook to provide tutorial
style documentation for MB-System Version 5. A preliminary (incomplete)
draft of this cookbook by Val Schmidt, Dale Chayes, and David Caress is
being made available here along with the data samples used to
illustrate the use of our software (see
links below).
- New MB-System Discussion Email List Established
(December 5, 2003)
- We have established an MB-System discussion email list to
facillitate communication among MB-System developers and users see
links below (see
links below).
- SURF format API released under GPL by ATLAS
Hydrographics GmbH (October 18, 2001)
- ATLAS Hydrographics GmbH of Bremen, Germany, has released the
source code used to read and write swath mapping sonar data in the SURF
format as open source under the GNU Public License (GPL).
- ATLAS markets a number of widely used multibeam sonars,
including the Fansweep 15 and Fansweep 20 multibeams for shallow water
surveying and the Hydrosweep MD-2 and Hydrosweep DS-2 multibeams for
mid- to deep-water surveying.
- The ATLAS HYDROMAP processing package stores Fansweep
and Hydrosweep data in the SURF format. A large amount of seafloor
mapping data worldwide is archived in this format.
- We will be adding support for the SURF format in MB-System
soon (as time and resources allow).
- MB-System NSF Proposal Funded(April
6, 2001)
- The Marine Geology and Geophysics Program of the NSF has
approved our request for another five years of funding for MB-System
development and support.
- We thank all of the MB-System users who provided letters of
support for inclusion in this proposal. Your time and effort are
appreciated!!
- We also thank Dr. David Epp, our NSF Program Manager, and the
mail reviewers and panel members who reviewed our latest proposal.
- Caution
- MB-System is under active development and is updated periodically. This
(usually) results in improved performance and additional capabilities,
but often means that the documentation (including these web pages) lag
somewhat behind the actual software. Please bear with us and send your
comments and suggestions.
- Download and Install MB-System
Complete documentation about downloading and installing MB-System is available at:
The MB-System source code distribution and related files, including
data samples, are available by anonymous ftp to
Click here to download (by ftp) the most recent MB-System source code release.
Packaged MB-System Distributions
MB-System is included is several package manager systems for Mac and Linux repositories, providing an easy means for installing our software. The biggest advantage of using package managers is that the prerequisite packages are identified and installed automatically. In most cases these packages are not prepared and maintained by MB-System developers, and generally the versions available in external packages lag those available from us through source distributions (particularly in the repositories described as stable). We think that the majority of MB-System users now install the software through one of these package managers.
Poseidon Linux
MB-System is included in Poseidon Linux, the "Scientific GNU/Linux". Poseidon is an Ubuntu Linux based distribution including a selection of open source software packages intended to support scientific research. In particular, it offers several specific tools in the areas of GIS, 3D Visualization, Mathematics, Statistics and several other fields of research.
UbuntuGIS for Ubuntu Linux
MB-System is one of the packages maintained in the UbuntuGIS repository.
Fink for MacOs
MB-System has been packaged for MacOS X as part of the Fink Project. The MB-System Fink package can be found at:
http://pdb.finkproject.org/pdb/package.php/mbsystem
If the fink package manager is installed, then MB-System and its prerequisites can be installed using this command:
fink install mbsystem
HomeBrew for MacOs
MB-System has also been packaged for MacOs X as part of Homebrew.
If you have Homebrew installed, then MB-System and its prerequisites can be installed using these two commands:
brew tap homebrew/science
brew install mbsystem
OSGeo-Live Distribution
The OSGeo-Live
distribution includes MB-System. OSGeo-Live is a self-contained bootable DVD, USB thumb drive or Virtual Machine based on Lubuntu that includes a wide variety of open source geospatial software.
Acknowledgements
Thanks to Christian Ferreira for creating and distributing Poseidon Linux, to Johan Van de Wauw for updating the MB-System package in UbuntuGIS, to Kurt Schwehr for maintaing the Fink MB-System package, to Anthony Lukach, Simon Dreutter, and Andrew Janke for maintaining the Homebrew MB-System package, and to Hamish Bowman for maintaining the OSGeo-Live MB-System package.
Documentation
-
The complete web page documentation distributed with MB-System can be
accessed here:
Current Release:
-
The MB-System Cookbook tutorial
documentation by Val Schmidt, Dale N. Chayes, and David W. Caress is
available both as web pages and as a downloadable pdf document:
A document providing guidance for integrating a third-party
bathymetry
editor with the MB-System
processing environment can be accessed here:
A document providing guidance for supporting new data formats in MB-System
can be accessed here:
- A few swath data format specifications made available to us can
be
viewed here:
Documents concerning how mapping sonars work can be viewed here:
MB-System User's and Developer's Discussion
Lists
We maintain two MB-System email discussion lists to facillitate
communication among MB-System users and developers.
The MB-System User's Discussion List is intended for questions about how to use MB-System, for
discussions of bugs and other problems, and for suggestions about improving
the software. We encourage users
with questions and/or problems to use this list rather than emailing
the developers directly. One must subscribe to the list in order to
post messages, but the message archive is publically viewable and
searchable over a web interface. The maximum message size in this list
is 100K.
The web interface to the MB-System User's Discussion List is at:
http://listserver.mbari.org/sympa/info/mbsystem
To subscribe to the MB-System User's Discussion List go to:
http://listserver.mbari.org/sympa/subscribe/mbsystem
To read the MB-System User's Discussion List archives go to:
- http://listserver.mbari.org/sympa/arc/mbsystem
The MB-System Developer's Discussion List is intended for detailed discussions amongst the active MB-System developers.
All are welcome to join the developer's list, but be warned that topics may not be of general interest. One must subscribe to the list in order to post messages, but the message archive is publically viewable and
searchable over a web interface. The maximum message size in this list
is 100K.
The web interface to the MB-System Developer's Discussion List is at:
http://listserver.mbari.org/sympa/info/mbsystem-dev
To subscribe to the MB-System Developer's Discussion List go to:
http://listserver.mbari.org/sympa/subscribe/mbsystem-dev
To read the MB-System Developer's Discussion List archives go to:
- http://listserver.mbari.org/sympa/arc/mbsystem-dev
- MB-System Announcement List
We maintain another very low traffic list of email addresses which is
used solely for providing notification of major updates. If you would
like to be notified when new releases are available, but do not wish to
join the discussion list described above, please send an email message
to:
- majordomo@ldeo.columbia.edu
- with a single line in the body of the message that says:
- to be automatically added to the announcement list.
- Last Updated: 23 October 2016