ValiDat.DSM, a new soil data validation dataset for Central Europe 313 B r i e f i n f o r m a t i o n Hungarian Geographical Bulletin 62 (3) (2013) 313–320. ValiDat.DSM, a new soil data validation dataset for Central Europe Endre DOBOS1, Erika MICHÉLI2, Emil FULAJTÁR3, Vít PENÍŽEK4 and Marcin ŚWITONIAK5 Abstract Digital soil maps are oft en derived using digital soil mapping tools, satellite imageries and digital terrain models as environmental covariates. Therefore several new datasets are raster based data representing soil classifi cation categories, like WRB reference soil groups. Validating raster datasets with categorical data is not well researched and supported. No procedure and validation datasets exist that can take categorical diversity and similarity (taxonomic distance) into consideration. This approach would require an input validation dataset describing the categorical diversity of the spatial units to be validated. The aim of this study is to introduce a novel dataset developed for this purpose. Keywords: ValiDat.DSM, soil validation, DSM, raster dataset, categorical data Introduction Digital soil mapping has become a very effi cient tool in soil science, and several applica- tions have been published (McBratney, A.B. et al. 2003; Lagacherie, P. et al. 2006). Many of these applications use environmental covariates like remotely sensed images and digital elevation models, which are raster based data sources with block support. Raster format is favoured by the many users as well. The majority of soil data users require data in raster format with values of certain properties, like pH, clay content or soil organic 1 Institute of Geography, University of Miskolc. H-3515 Miskolc-Egyetemváros, Hungary. E-mail: ecodobos@uni-miskolc.hu 2 Department of Soil Science and Agrochemistry, Szent István University. H-2100 Gödöllő, Páter Károly u. 1. Hungary, E-mail: micheli.erika@mkk.szie.hu 3 Soil Science and Conservation Research Institute. Gagarinova 10, Bratislava, 827 13, Slovakia. 4 Czech University of Life Sciences Prague. Kamýcká 129, Praha, 165 21, Czech Republic. 5 Nicolaus Copernicus University. Gagarina 11, Torun, 87–100, Poland. 314 matt er content. Qualitative data can be later classifi ed and used as categorical data. The most typical categorical soil data is the soil type/classifi cation category, like WRB (IUSS Working Group WRB, 2006) or the national classifi cation systems. Pixels represent a homogeneous spatial object having only one descriptive value or class allocated to it. However, the land surface area represented by a pixel has a more or less heterogeneous soil coverage. This heterogeneity is diffi cult to handle in a “one value environment“. The quantitative variables oft en use the average value, while the categorical variables use the dominant class of the pixel area. Both methods simplify the real heterogeneity of the area. Quantitative informa- tion can be further explained by descriptive statistics, like standard deviation, minimum, maximum, range etc. Explaining the diversity is more diffi cult for the categorical data. A po- tential way to characterize the pixel area is the fuzzy membership approach, when each po- tential class is represented by a corresponding layer representing the occurrence likelihood or spatial share of the given soil class within the pixel (A-Xing Zhua, et al. 2010; De Gruij ter, J.J. and McBratney, A.B. 1988; McBratney, A.B. and Odeh, I.O.A. 1997; McBratney, A.B. et al. 1992, 2000). This is an appropriate way to keep the heterogeneity information, but user do not prefer this way of information presentation due to its data complexity. Fuzzy data sets are oft en simplifi ed in the preprocessing steps by selecting the one with the highest share - namely the dominant class - and the rest of the information is lost. The presentation and validation of the raster based, categorical soil data is not well developed. The e-SOTER project developed a novel approach to present categorical information on block support. The resulting dataset has several layers of occurrence prob- abilities of WRB diagnostic horizons/features/properties and an additional layer of the reference soil group (RSG) of the WRB system (IUSS Working Group WRB, 2007). However, no appropriate validation methodology and data exist so far. This paper describes a novel approach for the development of a validation da- tabase, entitled as ValiDat.DSM and its potential use for validating digital soil mapping derived WRB reference soil groups and the occurrence probabilities of selected diagnostics. The sampling methodology combines an automated simple random sampling with slight adjustment for bett er accessibility and fi t to the raster database and a systematic random sampling approach to populate the selected pixels with additional observations. Methods Overall validation procedure An external validation dataset was developed for predicting the accuracy of categorical raster soil datasets. The ValiDat.DSM dataset has 114 validating sites from the four Visegrád Countries: 17 from the Czech Republic, 58 from Hungary, 23 from Poland and 16 from Slovakia (Figure 1). The sites/pixels for validation were randomly selected. All sites had 5 observations falling within a 450 by 450 meters pixel area. Having these 5 observations, proportions of the RSG within the pixel can be approximated with 20%, 40%, 60%, 80% and 100% coverage. The coordinates of the sites are given in WGS_1984_UTM_Zone_34N projection system (Projection: Transverse Mercator, False Easting: 500,000, False Northing: 0, Central Meridian: 21, Scale Factor: 0.999600, Latitude of origin: 0, Linear Unit: Meter, Datum: D_WGS_1984). 315 Fi g. 1 . T he p os iti on o f t he v al id at io n si te s 316 Field work and database design Each sites had one profi le opened in the centre of the selected pixel. This soil pit was described and all WRB diagnostic criteria, materials, horizons and features have been documented and the classifi cation name was defi ned. Chemical properties were identifi ed using only fi eld tools, like HCl 10% solution for CaCO3 content, pH indicators, alpha-alpha-dipiridil test for free iron detection. The interpreta- tion and the translation of the results into quantitative information were done using expert knowledge and the soil description guidelines of the FAO (FAO, 2006). Field work for the core set (65 profi les) was done by an international expert group representing all four countries. Four additional augerings were deepened 100 m North, East, South and West from the pit. The material taken out from the hole has been put into a 1 m long tray keeping the original depth. By this way a disturbed profi le has been created and was taken back to the pit, where all four were put next to each other in a clockwise order starting from North. Documenting photos were taken from the trays and the pit as well (Photo 1a, b). All four dis- turbed profi les have been described in the same way as the pit. In some cases, where the disturbed material did not let us recognizing the diagnostic the features important for the classifi cation (like the lamellas or clay coatings), the existence or lacking of them was assumed based on the pit description. At the end a table was compiled with fi ve observations and all diagnostic properties, features, horizons and material have been listed for each of the observations (Table 1). Based on the fi ve observations per site, a table with the RSG classes and diagnostics were listed with an appropriate proportion rounded up to 20%, like 20%, 40%, 60% 80% and 100% (Table 2). Photo 1a, b. Standard photos of the profi les and the four augerings. The soil trays from left to right are in clockwise order starting from North (N–E–S–W respectively) 317 Ta bl e 2 . T he in te rp re te d va lid at io n da ta se t f or se ve n pr ofi le s i n H un ga ry Pr ofi le ID C o- or di na te s C la ss p ro ba bi lit y* % W RB R ef er en ce S oi l G ro up p ro ba bi lit y x y G le yi c- St ag ni c- Re du ci ng c on di tio n M ol lic H or iz on C al ci c H or iz on C al ci c H or iz on (C al ci so l) D ys tr ic Eu tr ic RS G % 1 2 3 4 5 6 7 56 79 35 52 82 19 51 08 72 44 69 61 43 23 53 39 49 20 42 04 84 53 11 60 1 52 81 92 8 51 77 84 5 52 06 60 5 52 10 71 3 51 93 36 6 51 68 25 8 – – 10 0 – – – 40 – 10 0 10 0 10 0 10 0 – 80 – 10 0 10 0 10 0 10 0 – 80 – – – – – 80 20 10 0 – – – – – – – 10 0 10 0 10 0 10 0 80 10 0 A re no so l C he rn oz em C he rn oz em C he rn oz em C he rn oz em C al ci so l A re no so l C he rn oz em C al ci so l 10 0 10 0 10 0 10 0 10 0 80 –2 0 80 –2 0 * V al ue s fo r S po di c, A rg ic , C am bi c, V er tic (V er tis ol ), Sa lic a nd N at ri c H or iz on s w er e 0% . Ta bl e 1 . A n ex am pl e o f t he v al id at io n da ta se t* C ol ou r pH to p Te xt ur e C aC O 3 D ia gn os tic ho ri zo ns , pr op er tie s, m att e ri al s W RB n am e N or th a nd E as t So ut h an d W es t D ia gn os tic ho ri zo ns , pr op er tie s, m att e ri al s W RB n am e D ia gn os tic ho ri zo ns , pr op er tie s, m att e ri al s W RB n am e A p: 0 –2 0 cm : 1 0Y R 5/ 4 Bt /C : 2 0– 80 c m 1 0Y R 5/ 6 La m el la : 1 0Y R 4/ 4 4– 5 – – Sa nd – – 0 – – A re ni c D ys tr ic La m el lic La m el lic A RE N O SO L (D ys tr ic A re ni c D ys tr ic La m el lic La m el lic A RE N O SO L (D ys tr ic ) A re ni c La m el lic D ys tr ic La m el lic A RE N O SO L (D ys tr ic ) * C ou nt ry ID : H U ; P ro fi l e ID : 1 ; S ite n am e: A pa gy ; C oo rd in at as . x = 5 67 93 5; y = 5 31 16 01 318 100% was given for a certain diagnostic, when it could be found in all observations, while 40% was given when 2 out of the fi ve showed the certain feature. The RSG column lists all RSG observed in the site having the proportion list as well, where the proportions are rounded in the same way as for the diagnostics and sums up to 100% to a site. Site selection methodology The sites have been selected randomly. These sites had to be moved to the closest pixel centres and checked for accessibility, potential disturbance or other restricting factors. The whole site optimization procedure was programmed in ArcGIS, no personal bias could have a signifi cant impact on the site selection. Required input data: randomly generated sampling points by ArcGIS random point generator; the raster dataset to be validated (in this case the e-SOTER Central European window); vector-based GIS databases of the sett lements, road and railroad networks, water bodies, nature conservation and other protected areas of the country. The site location optimization process A 5 pixel circle shape neighbourhood around the selected point was selected as potential sampling pixels. Because neither the pits, nor the auger sites should be within sett lements or on roads, railroads, or any similar locations or even close to them, a 50 m limit was set as minimum distance from the lines or polygons symbolizing them in the vector databases. This limit was increased to 150 m because the auger sites are 100 m far from the profi le pit, so to keep the minimum 50 m distance in the case of the auger sites, the pit should be at least 150 m far from the excluded areas. Every points falling within a distance of 150 m from roads, railroads, or sett lements were deleted from the possible sampling points, just like the points that were closer to the water bodies, or protected areas then this limit. At the end an accessibility test was performed on the data. A 500 m maximum al- lowed distance was set up from the closest road to make sure that the fi eld sampling group does not have to spend too much time on approaching the points and transport the gears there. These two steps of fi ltering result a set of potentially selectable pixels. The closest to the original randomly selected point was selected as validation pixel. Results and discussion Validation of categorical information, like WRB RSG, is a complex problem. Congalton, R.G. (1991) and Brus, D.J. et al. (2011) reviewed the most common tools and approaches. Taxonomic adjacency or genetic relationship within a certain set of soil forming factors makes a signifi cant diff erence in the level of misclassifi cation (Phillips, J.D. 2013). Misclassifying a pixel to a related RSG or to a “nonsense” RSG does not mean the same level of uncertainty. Minasny, B. and McBratney, A.B. (2007) have published an approach to quantify the diff er- ences between the soil classes by estimating the taxonomic distances for the WRB RSG classes. This approach is very promising to solve the problem of taxonomic adjacency and quantify – – – 319 the taxonomic diff erences. However, the variables and their weights used to calculate the taxonomic distances are needed to be further refi ned for a more realistic picture. Besides of the lack of an advanced procedure for validation, the most limiting factor is the lack of appropriate, unbiased datasets describing the within-pixel variability, that can be used as ground truth for the validation. The aim of the ValiDat.DSM is to support new initiatives to develop a more appropriate and “standardisable” way of categorical soil data validation. The ValiDat.DSM dataset has three major forms of information. Table 1. shows the fi eld recording sheet. It describes the profi le physical and chemical properties needed for the WRB classifi cation procedure and all diagnostics that was identifi ed in the profi le and the offi cial WRB classifi cation category. The second half of the table records all diagnostics for the four augerings done 100 m North, East South and West from the profi le and also the WRB classifi cation names. This table can be used to understand the site when data used for scientifi c purposes. Table 2. is derived from table 1 by interpreting the soil variability expressed in a selected set of diagnostics important in Central Europe and by the WRB RSG (Dobos, E. et al. 2010, 2011, 2013). The tabulated information is complemented with soil profi le photos and photos on the landscape and the four auger sites in one picture. This latt er one is a magnifi cent tool for soil diversity representation. This information can be used as fi eld/ground truth data for validating soil categori- cal information with estimated proportions or occurrence probabilities. Having informa- tion on the spatial share of the soil classes within the pixels, advanced techniques can be used to assess the real reliability of the datasets. The validation can be done considering the taxonomic adjacencies/distances (Phillips, J.D. 2013; Minasny, B. and McBratney, A.B. 2007) between the WRB RSG classes and defi ning similarity factors to express their relationship in the quantifi cation of the level of misclassifi cation/uncertainty. This dataset can be used for research purposes as well for soil variability studies within diff erent soil forming environments important for soil mapping and for the defi nition of the minimum set of sampling sites for mapping and validation. Conclusions The ValiDat.DSM dataset has been initiated for Central Europe with the contribution of four countries, Czech Republic, Hungary, Poland and Slovakia. The dataset is freely available aft er registration in the project site (htt p://www.uni-miskolc.hu/~soil/index.html). Data is presented there in several ways, excel sheet format and the documenting sets of photos and in several kind of GIS environment for visualization helping the users understanding the spatial relationships. The dataset is a good tool for validating DSM derived soil datasets and for scien- tifi c researches on soil variability within diff erent soil forming conditions. At the end each validation sites – pixel area – have 5 observations. Therefore the overall purity – defi ned as the proportion of the mapped area covered by a certain soil class – can be predicted. Acknowledgements: Our work has been supported by FP7 project “Regional pilot plat- form as EU contribution to a Global Soil Observing System” Grant agreement no.: 211578, fi nanced by the European Commission“; by the Hungarian National Scientifi c Research Foundation (OTKA, Grant No. K105167); by the „Validation of the Central European Soil database” Strategic Grant of the Visegrad Fund, No. 31210072, and by the BONUS- HU Grant No. OMFB-01251/2009 and by the “Excellent Research Faculty” Grant of the Hungarian Ministry of Human Resources” (registration no.: 17586-4/2013/TUDPOL). 320 REFERENCES A-Xing, Z., Lin, Y., Baolin, L., Chengzhi, Q., Tao, P. and Baoyuan, L. 2010. Construction of membership functions for predictive soil mapping under fuzzy logic. Geoderma 155. (3–4): 164–174. Brus, D.J., Kempen, B. and Heuvelink, G.B.M. 2011. Sampling for validation of digital soil mapping. European Journal of Soil Science 62. 394–407. Congalton, R.G. 1991. A review of assessing the accuracy of classifi cations of remotely sensed data. Remote Sensing of Environment 37. 35–46. De Gruij ter, J.J. and McBratney, A.B. 1988. A modifi ed fuzzy k-means method for predic- tive classifi cation. In Classifi cation and Related Methods of Data Analysis. Ed.: Bock, H.H. Amsterdam, Elsevier, 97–104. Dobos, E., Seres, A., Vadnai, P. 2010. Az e-SOTER digitális talajtérképezés módszertana (Methodology for digitalized soil mapping “e-SOTER”). V. Magyar Földrajzi Konferencia. Pécs. 4–6. Nov. University of Pécs. Manuscript. Dobos, E., Seres, A., Vadnai, P., Michéli, E., Fuchs, M., Láng, V., Bertóti, R.D. and Kovács, K. 2013. Soil parent material delineation using MODIS and SRTM data. Hungarian Geographical Bulletin 62. (2): 133–156. Dobos, E., Vadnai, P., Michéli, E., Láng, V., Fuchs, M. and Seres A. 2011. Új generációs nemzetközi talajtérképek készítése, az e-SOTER módszertan (Making international soil maps of new generation, methodology “e-SOTER”). Térinformatikai Konferencia. Debrecen. 19–20 May, 2011. University of Debrecen. Manuscript. FAO, 2006. Guidelines for soil description. Rome, FAO. IUSS Working Group WRB, 2006. World reference base for soil resources 2006. 2nd edition. World Soil Resources Reports No. 103. Rome, FAO, IUSS Working Group WRB, 2007. World reference base for soil resources 2006, update 2007. 2nd edition. World Soil Resources Reports No. 103. Rome, FAO. Lagacherie, P., McBratney, A.B., and Voltz, M. eds. 2006. Digital soil mapping: an introduc- tory perspective. Amsterdam, Elsevier. 600 p. McBratney, A.B. and Odeh. I.O.A. 1997. Application of fuzzy sets in soil science: fuzzy logic, fuzzy measurements and fuzzy decisions. Geoderma 77. (1–2): 85–113 McBratney, A.B., de Gruij ter, J.J. and Brus. D.J. 1992. Spatial prediction and mapping of continuous soil classes. Geoderma 54. (1–2): 39–64. McBratney, A.B., Mendonça-Santos, M.L. and Minasny, B. 2003. On digital soil mapping. Geoderma 117. (1–2): 3–52. McBratney, A.B., Odeh, I.O.A., Bishop, T.F.A., Dunbar, M.S. and Shatar. T.M. 2000. An over- view of pedometric techniques for use in soil survey. Geoderma 97. (3–4): 293–327. Minasny, B. and McBratney, A.B. 2007. Incorporating taxonomic distance into spatial prediction and digital mapping of soil classes. Geoderma 142. (3–4): 28–293. Phillips, J.D. 2013. Evaluating taxonomic adjacency as a source of soil map uncertainty. European Journal of Soil Science 64. 391–400. << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Error /CompatibilityLevel 1.3 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize false /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments true /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /CreateJDFFile false /Description << /ARA /BGR /CHS /CHT /CZE /DAN /DEU /ESP /ETI /FRA /GRE /HEB /HRV (Za stvaranje Adobe PDF dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. Stvoreni PDF dokumenti mogu se otvoriti Acrobat i Adobe Reader 5.0 i kasnijim verzijama.) /ITA /JPN /KOR /LTH /LVI /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR /POL /PTB /RUM /RUS /SKY /SLV /SUO /SVE /TUR /UKR /ENU (Use these settings to create Adobe PDF documents best suited for high-quality prepress printing. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) /HUN >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /ConvertToCMYK /DestinationProfileName () /DestinationProfileSelector /DocumentCMYK /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice