GeoEasy an open source project for surveying calculations GeoEasy an open source project for surveying calculations Zoltan Siki Department of Geodesy and Surveying, Budapest University of Technology and Economics in Budapest, Hungary siki.zoltan@epito.bme.hu Abstract. GeoEasy is a complex tool for land surveyors to calculate coordinates from ob- servations. It is available on Linux and Windows platforms, too. A graphical user interface (GUI) makes the program user friendly. Besides the basic surveying calculations like inter- section, resection, traversing, etc. modules extend its functionality to network adjustment, digital terrain models and regression calculations. The program supports several input and output formats, so it can easily be inserted into user’s work-flow. This paper introduces some functionalities of the software. From version 3.0 GeoEasy became free software under GPL license. Volunteers (testers, document writers, developers) are welcome! Keywords: surveying calculation; network adjustment; DTM; regression calculation; open source. 1. Introduction Though GeoEasy is quite new among the open source software, it has a longer history as a proprietary program. In the late nineties there were demands for a software for surveying calculations which is Hungarian localized. At that time Tcl/Tk (Tool Command Language/- ToolKit) [1] script language was a popular graphical programming environment working for both Linux and Windows platforms, so this language was chosen in 1997 for GeoEasy, the software which I started to develop for surveying calculations. During the last twenty years with less or greater intensity the development continued. Nowadays many Hungarian small enterprises and our university use it. In 2017 it became free software under GPL license, freely available for everybody. The source code repository can be found on GitHub [9], bi- nary releases for Windows (32/64 bit) and Linux (64bit only) can be downloaded from the home page of the project [5]. 2. Base conception Objectives of the development were to create a software with user friendly graphical user interface (GUI) for surveyors in a modular structure with flexible, open connections to other programs. Educational purposes were also considered beside productivity. There are step by step and automatic (several calculation made in a single step) solutions for the same task. For exam- ple users can calculate orientations station by station manually selecting from the possible backsight directions or all orientations can be done in a single automatic process. Horizontal coordinates are supposed to be plane coordinates in the same projected Coordinate Reference System (CRS). GeoEasy utilizes other open source projects and several open data formats. GNU Gama [3, 2] is used for geodetic network adjustment, Triangle [8] is used to generate triangulated irregular Geoinformatics FCE CTU 17(2), 2018, doi:10.14311/gi.17.2.1 1 http://orcid.org/0000-0002-9615-181X https://doi.org/10.14311/gi.17.2.1 http://creativecommons.org/licenses/by/4.0/ Z. Siki: GeoEasy an open source project for surveying calculations networks (TIN). These programs are used through file data exchange. GeoEasy has interfaces to these CLI programs which generate the input file(s) for the external tools and read back their output. Other open source tools are used to create binary releases and install kits. Tcl Cruncher generates the compressed Tcl source code, removing comments, FreeWrap [6] wraps the Tcl code into an executable Linux/Windows file, Nullsoft Scriptable Install System (NSIS) [7] makes the windows install kit. The possible import/export formats are listed in Figure 1. All import and export formats use different text files, which are platform independent. Some of the input files come from total stations (Leica GSI, IDX; Sokkia SCR, SDR; TopCon 700, 210; Trimble M5; Geodimeter Job/Are). The other input files can come from other programs (Surv CE RW5, AutoCAD DXF, Excel CSV, ArcGIS ASCII GRID, n4ce). The imported field-books can be postpro- cessed and edited in GeoEasy. The export formats on one hand can be used to upload coor- dinates to the instrument in a vendor specific format. On the other hand the export formats are useful to import data into other programs (AutoCAD, GoogleEarth, GPS Trackmaker). Figure 1: Data import/export formats. An extendable, flexible native data structure was developed for GeoEasy to store field-book and coordinate data. It is based on Tcl lists and associative arrays. Each field-book is an array, an array element contains station or observation data for a point. The order of observations are kept by integer array indexes. Each array item is a list of sublists, a sublist contains two items, the first is an integer code and the second is a value for that code (see Figure 2). The codes are taken from Geodimeter Job/Are format and extended by necessary extra codes. An observation record must have 2 (station ID) or 62 (backsight ID) or 5 (point ID) codes. Other code/value pairs are optional, for example 3 (instrument height), 6 (signal height), 7 (horizontal angle), 8 (vertical angle), 9 (slope distance). During the development new codes were added without refactoring the program code. A very similar structure is used for coordinate lists. The only difference is in the array indexes Geoinformatics FCE CTU 17(2), 2018 2 Z. Siki: GeoEasy an open source project for surveying calculations Figure 2: Field-book data structure. and the used codes. This case associative arrays are used where the index is the point id, which must be unique in the array. A third type of data is the parameter file. It is a single list of code value pairs defining some general information for a field-book (e.g. date, user name, etc.). Several data sets can be loaded and used for the calculations in a session, which are handled to be in the same spatial reference system. The currently loaded field-books and coordinate lists are stored in memory. The coordinate lists can contain two types of coordinates, preliminary and final coordinates. Preliminary coordinates are used not only for network adjustments but to display points in the graphic window. This way the user can visualize and select point(s) in the graphic window before deciding the coordinate calculation method(s). The GUI consists of several windows which is usual on Linux but can be unusual for Win- dows users. It more resembles to GIMP (GNU Image Manipulation Program) user interface. Starting the program two windows will be opened, the main window contains a menu and few graphics, and the calculation results window will hold the calculation results and program log messages. Users can open separate windows to edit field-books and coordinate lists, to display points and observations in a map view (see Figure 3). The program can handle several templates to visualize field-books and coordinate lists. The templates consist of column definitions which include codes for values to display and units to convert to. Beside the build in templates user defined templates can be added, too. DMS, GON (GRAD), DEG angle units and METER, FEET, FATHOM length units are supported. Users can change, extend standard mask definitions or can create separate files for custom mask definitions and load them on demand. Finally it would be mentioned among base concepts that calculations are repeatable. After Geoinformatics FCE CTU 17(2), 2018 3 Z. Siki: GeoEasy an open source project for surveying calculations Figure 3: Different windows of the program. changing the observations, coordinates or parameters users can repeat any calculation, but it is not automatic. Calculation results are sent to different targets. Calculation details can be seen in the calculation results window if it is open and always sent to the log file (geo_easy.log) in the installation directory. 3. Using the software 3.1. The user interface The user interface of the program consists of menus, sub-menus of the different windows and modal dialog boxes. Some types of windows, the field-book, the coordinate list and the graphic windows have pop-up menus, too. The graphic windows also have a toolbar. Before starting the work in GeoEasy data have to be loaded. Input sources are: • Electronic field-book downloaded from a total station • Text file created by other programs • Manual data input through the GUI Any number of GeoEasy data sets from different sources can be opened in the same session. The user can change the used template (displayed columns and units) for the opened field- book, for example the angle units can be changed from DMS to GON changing the template. The content of the field-books and coordinate lists can be edited. Users can open more (maximum 10) graphic windows. Points from all loaded data sets are displayed in a graphic window. Besides point markers and point IDs the observations are displayed as lines, an arrow shows the direction of the measurement. The fill color of the point markers have a special meaning, green means oriented station, red not oriented station, Geoinformatics FCE CTU 17(2), 2018 4 Z. Siki: GeoEasy an open source project for surveying calculations white no station. Point ID label is red if only preliminary horizontal coordinates are available for a point. The content of the graphic windows can be set on a window base and the used colors can also be customized. 3.2. Simple calculations Most of the basic coordinate calculations are involved in GeoEasy. There are calculations for the coordinates of a single point (e.g. intersection, resection). These calculations can be selected from the pop-up menu, right click on the point in any window and select from the different calculation methods. Those methods which are not available (e.g. there are not enough observations) are greyed out. Calculation methods which calculate the coordinates of more points (for example traversing) can be found in the Calculation menu. Traverse lines can also be given in the graphic window using the Traversing tool from the toolbar. Two dimensional coordinate transformations are also available in GeoEasy and are based on common points in two projected spatial reference systems. These transformations can be used only for a limited area of the surface of the Earth, usually few square kilometers. Transformation is made between a source data set, which has to be loaded and a selected target data set. Depending on the number of common points in the two data sets with horizontal coordinates, users can select orthogonal (three or four parameters), affine and polynomial (second and third order) transformations. Not only the transformed coordinates but the transformation parameters can also be saved and can be applied for other data sets. In the result list of a transformation the residuals and RMS are calculated. 3.3. Network adjustment Most of the work in network adjustment is done by GNU GaMa. GeoEasy only prepares the data for adjustment. All loaded data sets are considered in the adjustment calculations. Though gama-local can calculate preliminary coordinates, in GeoEasy the preliminary co- ordinates and orientations have to be calculated before network adjustment. It can be an automatic process, Preliminary coordinates from the Calculation menu. The aim of this is to give more control into the hand of the user. On one side points with preliminary coordinates can be seen in the graphic window, users can visually check the shape of the network before adjustment. On the other side GeoEasy can compare the observed values to the calculated values from the preliminary coordinates before adjustment and sends warn- ing if the difference is too large. Preliminary coordinate calculation is made in an iteration. First approximate orientations are calculated, in this case points with preliminary coordi- nates are also considered as backsight directions. Than different trigonometric calculations (polar, resection, intersection, arcsection, etc.) are used for coordinate calculations. After new coordinates are calculated for a point, new orientation calculations are tried again, fol- lowed by coordinate calculations again, until no more new orientations and coordinates can be calculated. Similar iteration is used for preliminary elevations. At the end of the process users get a list of points, what horizontal coordinates or elevations cannot be calculated for. A priori standard deviation of observations can be set by data sets in the dialog box of the Observation parameters from the Edit menu of the main window or globally in the dialog box of the Calculation parameters from the File menu of the main window. Global a priori standard deviations are only used if not set in the calculation parameters of the data Geoinformatics FCE CTU 17(2), 2018 5 Z. Siki: GeoEasy an open source project for surveying calculations set. Standard deviations for short directions are handled specially if the Distance limit value is set in the dialog box of the Adjustment parameters from the File menu of the main window. If mean direction or zenith angle is measured between points closer to each other than the limit, the standard deviations are increased (multiplied) by the ratio of the distance limit and the distance of the two points. The default value for distance limit is 200 meters. Users can select 1D/2D/3D network adjustment from the Calculation menu. Observations which are related to a point without coordinates are not sent to GNU GaMa XML file for adjustment. Free network is calculated if all points are marked as unknown. Adjusted coor- dinates and orientation angles are read back from GNU GaMa XML output. Blunders marked by GNU GaMa are not automatically removed from adjustment. Users have to manually remove the observation with the largest statistic above critical from the field-book and run the adjustment again. Repeat this process until all blunders are removed. 3.4. Regression calculation Regression calculation is useful in geometrical control of engineering objects, structures. For example using observed points the following questions can be answered. Is the wall vertical or is the new tunnel circular? The best fitting geometry for the selected points can be calculated with this module using least squares method. Some linear and non linear shapes can be selected in 2D or 3D. All points are considered with unit weight in the adjustment calculation. Supported geometries are 2D line (three variants), 2D circle, general plane (two variants), horizontal plane, vertical plane, 3D line and sphere. Results are shown in the Calculation results window (see Figure 4) and stored in the log file. Besides the parameters of the geometry the distances from the best fitting shape to the given points and RMS are calculated. RMS (Root Mean Square) is a measure of the imperfection of the regression shape to the points. Figure 4: Circle regression results. Geoinformatics FCE CTU 17(2), 2018 6 Z. Siki: GeoEasy an open source project for surveying calculations 3.5. Digital Terrain Models (DTM) The DTM module generates Triangulated Irregular Networks (TIN) using the open source Triangle project. Not only scattered points but break lines can also be input for DTMs, so constrained Delaunay triangulation is made. DTM functionality is only available from the menu of the graphic window. Users can select from two input sources. One source is the points in the loaded coordinate lists and the break lines drawn in the graphic window. Other source can be an AutoCAD DXF file with 3D points and 3D break lines. The generated DTM is saved into three text files (.pnt, .pol and .dtm). Users can only load/use one DTM at a time. Triangles are displayed in the graphic window, too. Points, break lines and triangles can be deleted from a loaded DTM and point, break lines can be added to it. The TIN has to be regenerated after adding or deleting items to it. Besides the DTM generation this module adds more functionalities to GeoEasy. Volumes can be calculated between the TIN and a horizontal reference plane or between two TINs. In the later case GRIDs are generated from the two TINs for the volume difference calculation. Contour lines and profiles can be generated and exported to Autodesk DXF format. TINs can be exported into various formats (KML, ESRI ASCII GRID, VRML) to visualize them in 3D. 3.6. ComEasy The ComEasy module is responsible for the communication between the instrument and the computer. Jobs stored in the memory of the total station can be downloaded for data processing or coordinates can be uploaded for setting out on the field. Wired (RS-232) connections are supported, which are available on most total stations. From the beginning of the project it was released under open source license. 4. Localization Actually the GUI has English and Hungarian translations. English version is used in the English courses at our university. All messages are collected into a single Tcl source file which extension refers to the language (e.g. geo_easy.eng for English strings, geo_easy.hun for Hungarian strings). Users are encouraged to make a copy of the English message file and translate the strings to their mother tongue. There are about seven hundred messages to translate. Please send us back your translation or make a pull request on GitHub [9]. 5. Future plans The goal by changing the license to GPL was to widen the developing capacity and the user community behind the project. I was inspired by OSGeo projects which I took part in as a translator, documenter and coder. I hope developers, experts and users from all over the World will voluntarily be involved in the GeoEasy project. Issues and pull requests are accepted on the GitHub page [9] of the project. During the first two weeks, after changing the license to GPL, there were more than 200 downloads of the Windows installation kit. Geoinformatics FCE CTU 17(2), 2018 7 Z. Siki: GeoEasy an open source project for surveying calculations GeoEasy has a nephew project called SurveyingCalculation [10]. It is a QGIS plug-in, some parts of GeoEasy were rewritten to Python. This plug-in is available from the official QGIS plug-in repository. There is no clear future plan yet, how to maintain both projects parallel. Though the GUI of GeoEasy has been translated into English, the program contains some Hungarian specialties, for example the KLM and the GPS Trackmaker (TXT) export works for the HD72/EOV (EPSG:23700) spatial reference system only. In the development version on GitHub the cs2cs utility of proj.4 library is already used to convert coordinates from several spatial reference systems to WGS84, which is used by KML and GPS Trackmaker. The weakest part of the project is now the documentation. Previously only Hungarian doc- umentation has been made. The developer’s documentation [4], the installation guide and a step by step guide for the sample data sets are already available in English from the GitHub repository of the project. Acknowledgements Thanks to DigiKom Ltd to support the development of GeoEasy. I would also like to thank all colleagues and users of GeoEasy who helped me with suggestions and error reports. References [1] Ken Jones Brent B. Welch. Practical Programming in Tcl/Tk. Upper Saddle River, NJ 07458: Prentice Hall, 2003. isbn: 0-13-038560-3. [2] Aleš Čepek et al. GNU Gama. url: https://www.gnu.org/software/gama/ (visited on 02/06/2018). [3] Aleš Čepek. “The GNU GaMa Project - Adjustment of Geodetic Networks”. In: Acta Polytechnica 42.3 (Mar. 2002), pp. 26–30. url: https://ojs.cvut.cz/ojs/index. php/ap/article/download/350/182. [4] Developers documentation. url: http://digikom.hu/tcldoc/ (visited on 01/10/2018). [5] Download page for binary releases of GeoEasy. url: http://digikom.hu/english/ geo_easy_e.html (visited on 11/08/2017). [6] freeWrap. url: http://freewrap.sourceforge.net/ (visited on 11/08/2017). [7] NSIS (Nullsoft Scriptable Install System). url: https://sourceforge.net/projects/ nsis/ (visited on 11/08/2017). [8] Jonathan Richard Shewchuk. “Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator”. In: Applied Computational Geometry: Towards Geometric En- gineering. Ed. by Ming C. Lin and Dinesh Manocha. Vol. 1148. Lecture Notes in Com- puter Science. From the First ACM Workshop on Applied Computational Geometry. Springer-Verlag, May 1996, pp. 203–222. url: https : / / www . cs . cmu . edu / ~quake / tripaper/triangle0.html. [9] Source code repository of GeoEasy project on GitHub. url: https : / / github . com / zsiki/GeoEasy (visited on 11/08/2017). [10] Surveying Calculation module for QGIS. url: https://plugins.qgis.org/plugins/ SurveyingCalculation/ (visited on 11/08/2017). Geoinformatics FCE CTU 17(2), 2018 8 https://www.gnu.org/software/gama/ https://ojs.cvut.cz/ojs/index.php/ap/article/download/350/182 https://ojs.cvut.cz/ojs/index.php/ap/article/download/350/182 http://digikom.hu/tcldoc/ http://digikom.hu/english/geo_easy_e.html http://digikom.hu/english/geo_easy_e.html http://freewrap.sourceforge.net/ https://sourceforge.net/projects/nsis/ https://sourceforge.net/projects/nsis/ https://www.cs.cmu.edu/~quake/tripaper/triangle0.html https://www.cs.cmu.edu/~quake/tripaper/triangle0.html https://github.com/zsiki/GeoEasy https://github.com/zsiki/GeoEasy https://plugins.qgis.org/plugins/SurveyingCalculation/ https://plugins.qgis.org/plugins/SurveyingCalculation/ Z. Siki: GeoEasy an open source project for surveying calculations Introduction Base conception Using the software The user interface Simple calculations Network adjustment Regression calculation Digital Terrain Models (DTM) ComEasy Localization Future plans