why THE spatial metadata usecase not always succeeds

So what's THE spatial metadata usecase? Imagine you're looking for a nice map on butterflies in the Alpes region France. You open up your favourite map editor (be it ArcMap, QGis or an Openlayers webclient), be sure a metadata search plug-in is installed (http://webhelp.esri.com/geoportal_extension/9.3.1/index.htm#ext_csw_clnts.htm, http://nextgis.ru/en/blog/cswclient, http://gxp.opengeo.org/master/examples/catalogue.html). Imagine you already have the url for the french metadata portal (or to some portal harvesting all european/global datasets), be it http://www.geocatalogue.fr

Now when looking for keyword 'butterfly' and pointing to Alpes, in most cases you'll find a nice set of results, thanx to the CSW standard. But as soon as you want to add those datasets to a map the troubles start... Apparantly the standards (and/or implementations) are unfortunately not totally clear in describing the full use case. Here is a list of items which you'll encounter researching this use case in several implementations:

- CSW uses the Dublin Core standard to describe dataset metadata. There is however no clear way specified in DC how to put webservice (be it WMS, WFS or WMTS) information (like url, protocol) in the document. Apparantly a group of enthousast invented in 2006 a dclite4g profile on top of DC which introduced a key DC:Uri and DC:format http://wiki.osgeo.org/wiki/DCLite4G. Geonetwork still uses this DC:Uri. However without refering to the DCLite4G profile, resulting in a non-conformant DC-record. Inside the DC:Uri element attributes like protocol, name and description (from iso onlineresource) are available, which enables addition of the dataset-service to the map. The Geonode team recently proposed to solve the add2map challange by in stead of using a DC:Uri element, to use a DCT:references element (which is available in the DC Terms schema). The dct:references however has only one attribute, the attribute 'schema'. To be able to have parameters like protocal, name and url, they proposed to add a json object containing these values into the schema attribute (http://dev.geonode.org/trac/wiki/pycsw#MetadataLinks). Let's keep interoperability in mind while designing these standards-additions...

- A next challenge in this usecase is in the ISO online resource definition. Apparantly the standard doesn't describe where the layername is stored which presents the actual dataset. This results for example in ArcGIS CSW-client that all layers of a service are added to the map. Within the metadata community people have come to some form of an agreement to put the layername in the name element, and you'll see most of the current implementations use this agreement. For example the dutch government has put this agreement in their metadata profile (http://www.geonovum.nl/sites/default/files/nederlands_metadata_profiel_op_iso_19115_voor_geografie_-_v1.3.pdf).

I guess that's the reason they've put an option to choose your favourite flavour
of CSW 202 in ArcMAP CSW client

- Recent ISO standards offer more specified guidelines for creating these types of links. This approach however has quite some overhead, so won't get popular very soon, I haven't seen any client implemenation so far (besides the ones in Geonetwork and ArcGIS Portal). ISO proposed to add to each metadata for dataset record (iso:19115) a link to a number of metadata for service (iso:19119) records in which a layer presenting this data is available (WMS/WFS/WMTS). To get from metadata for dataset to the URL-properties one needs an additional CSW request.

- Apparantly the amount of requests needed to add a layer to the map needs to be minimised to keep up performance, especially in browser based applications like GeoExt. I saw some comments in the GeoExt/GXP code, that a Catalogue search implementation should not include a getcapabilities request to the actual mapserver. Probably because this is a very costly request taking quite some browser memory. But there is some information in that getcapabilities request you'll need for a seemless user experience: For WMS the projections are only in the getcapabilities, but also the available image formats and feature-info formats. One'll need to check them before being able to add the service to a map (which could have a different  projection/image format). This is especially true when adding a WMTS service (the tiling scheme is only available in getcapabilities). Getcapabilities request can turn really big as soon as 100's of layers are grouped in 1 service. So keep your services small. Latest Geoserver (2.2) supports splitting up the main service into multiple smaller services using workspaces. Or we might endorse OGC and the server vendors to also allow a layer parameter in a getcapabilities request. So you get a capabilities response for only the single layer you're interested in.

JeroenT, BartvdE, TomK, FrancoisXP, MarceldR Thanx for sharing these insights

Using comboboxes in an ExtJS PropertyGrid

When viewing/editing a database record in a PropertyGrid, you often encounter integer-foreign-keys referring to other tables containing lookup-labels. In the propertygrid you would want to display the lookup-label in stead of the integer. You could get the label with a json-ajax-request, but since you might need these values multiple time It's probaly better to have them cache in an ArrayStore.

var store = new Ext.data.ArrayStore({
fields: ["id", "label"],
data :  {"lookup":[[1,"blue"],[2,"red"],[3,"green"],[4,"yellow"]]}
});

Now you can reference the correct label from the ArrayStore in the customRenderer configuration of the PropertyGrid (ExtJS 3.4 implementation http://dev.sencha.com/deploy/ext-3.4.0).

 new ext.grid.PropertyGrid({
                            customRenderers: {
                                  //get the lookup label
                                'color': function(v){ return getLookup(store,v) }

                            },
                            propertyNames: {
                                'color':'Couleur' //set a localised property header
                            },
                            source: aRecordStore.record[0].data
                        })

function getLookup(store,id){
           if (store.find('id',id)==-1) return "-";
           else return store.getAt(store.find('id',id)).get('label');
}

Using a store here to retrieve a single label is not very optimal, but the store will come in handy when you want to edit the propertygrid. In the case of the foreign key, you would want to present the user with a dropdown (combobox) of available values:

new ext.grid.PropertyGrid({
                            customRenderers: {
                                'color': function(v){ return getLookup(store,v) } //get the lookup label
                            },
                            customEditors: {
                                //you don't want people to edit the id-field
                                'id': new Ext.grid.GridEditor(new Ext.form.TextField ({disabled : true})),
                                //get a combobox editor filled with lookup values to edit the color field
                               'color': new Ext.grid.GridEditor(new Ext.form.ComboBox({
                        store: store,
                    mode: 'local',
                    forceSelection: true,
                    triggerAction: 'all',
                    editable: false,
                    valueField: 'id',
                    displayField: 'label',
                }
            )
        ),
                            propertyNames: {
                                'color':'Couleur' //set a localised property header
                            },
                            source: aRecordStore.record[0].data
                        })

In case you want people to only be able to edit the propertygrid when a certain condition is met (is authenticated/edit button has been activated), use the beforeedit event:

listeners: {"beforeedit": function() {
                if(!readOnly) {
                    return true;
                } else {
                    return false;
                }
            }
}

Add dutch gridset to Geoserver 2.2

This weekend geoserver 2.2 got released. Besides the obvious novalties, like WFS2 support, new authentication system, able to connect to ldap, a nice feature has been added; the possibility to separate the workspaces as separate endpoints, each having it's own parameters (like title, abstract, point of contact, styles). To get the capabilities for just 1 workspace visit the url http://..../geoserver/workspace_name/wms?request.... To set individual parameters on a workspace check the 'settings' box in the workspace edit form.

Also new is the ability to create a gridset from within the geoserver admin interface, before only the gridsets epsg:4326 and epsg:900913 were available. Now you can easily create a gridset for any projection. The gridset is saved in the geowebcache.xml. For example the gridset for the dutch projection epsg:28992 would look like this (copy paste them in geowebcache.xml), according to the dutch tiling guidelines (by geonovum)

 <gridSet>
      <name>RD_NEW</name>
      <description>Scales indicated by geonovum in nederlandse richtlijn tiling v1.0 from june 2010</description>
      <srs>
        <number>28992</number>
      </srs>
      <extent>
        <coords>
          <double>-285401.92</double>
          <double>22598.08</double>
          <double>595401.92</double>
          <double>903401.92</double>
        </coords>
      </extent>
      <alignTopLeft>false</alignTopLeft>
      <resolutions>
        <double>3440.64</double>
        <double>1720.32</double>
        <double>860.16</double>
        <double>430.08</double>
        <double>215.04</double>
        <double>107.52</double>
        <double>53.76</double>
        <double>26.88</double>
        <double>13.44</double>
        <double>6.72</double>
        <double>3.36</double>
        <double>1.68</double>
        <double>0.84</double>
        <double>0.42</double>
        <double>0.21</double>
      </resolutions>
      <metersPerUnit>1.0</metersPerUnit>
      <pixelSize>2.8E-4</pixelSize>
      <scaleNames>
        <string>0</string>
        <string>1</string>
        <string>2</string>
        <string>3</string>
        <string>4</string>
        <string>5</string>
        <string>6</string>
        <string>7</string>
        <string>8</string>
        <string>9</string>
        <string>10</string>
        <string>11</string>
        <string>12</string>
        <string>13</string>
        <string>14</string>
      </scaleNames>
      <tileHeight>256</tileHeight>
      <tileWidth>256</tileWidth>
      <yCoordinateFirst>false</yCoordinateFirst>
    </gridSet>

custom workflow from featureinfo in GXP GeoEXT ExtJS

In a series of articles I would like to share some experiences with open source tooling we frequently use at GeoCat to fullfill customer wishes. I'd like to start with an article on the GXP library by OpenGeo. Developers at OpenGeo use this library in their famous products OpenGeoSuite, GeoNode and more to build interactive webbased mapviewers. The library is based on OpenLayers, ExtJS and GeoExt and can be used to build/configure a full interactive mapviewer including feature-query and edit in minutes. Download a copy from Github: https://github.com/opengeo/gxp.

At GeoCat we use the library if customers have very specific viewer demands which can not be met by a generic product like Geonetwork or Geonode. Today i did a small customisation which i think might be usefull to share with you, since a situation like this happens often. A customer wanted to display a list of coupled records from a non-geospatial table linked to a geometry a user clicked on in the map.

First challenge was to find an 'exit-point' from the default GXP workflow, in this case feature-info. I use exit point, because at that point you leave generic GXP workflow (display a record in a popup) and enter a customised workflow (open additional popups, ajax-requests and more). I wanted to convert the value of an attribute in the featureinfo result to a hyperlink, which starts up customised workflow. One way to set up such a hyperlink on an attribute is by addind a feature-template for that layer to geoserver and add the hyperlink in the template. Next choose html as display format for featureinfo (there is even another way available: one could add the hyperlink itself to the database, eg SELECT '<a href=' || www || '>click here</a>' AS link FROM ....). But in this case I added a ExtJS customRenderers on WMSGetFeaturinfo's itemConfig configuration. Such a renderer can override the default 'print as string' of a PropertyGrid to print about anything (like an image, hyperlink, ajax-lookup)

{
    ptype: "gxp_wmsgetfeatureinfo",
    outputConfig: {
        width: 400,
        height: 200
    },
    itemConfig: {
        customRenderers: {
            yourfieldname: function(value) {
                return '<a href='+value+'>Click here</a>';
            }
        }
    }
}

In stead of a hyperlink we'll need a javascript function call, to in this case load customised content and place it in a panel.

return '<button onclick="getPanelContent(' + value + ')"';

To retrieve the content from the PostGres database, i used PHP to generate JSON. Be sure to activate PostGres extension in PHP.ini.

<?php

$dbh = pg_connect("host=localhost dbname=postgis user=postgres port=5432");
$sql = "SELECT * FROM dummy where id =" + $_POST["id"];
$result = pg_query($dbh, $sql);   
$data = array();
while ($row=pg_fetch_object($result))
{
$data [] = $row;
}
echo json_encode($data);

?>

To retrieve the JSON I used an Ext.JSONStore with a load event that pushes the JSON result as a set of Ext.propertyGrids to a Panel with Accordion Layout (this is how GXP displays a usual getfeatureinfo result).

var store;

Ext.onReady(function(){

 store = new Ext.data.JsonStore({
    url: 'content.php',
    fields: [
            'Id','title', 'type'
        ],listeners: {
            load: {
                fn: function(store, records, options){
                 store.each(
                    function(record){
                     Ext.getCmp("accordionPanel").add(
                            new Ext.grid.PropertyGrid({
                            columns: [
                                {id:'Id',header: 'ID',   dataIndex: 'Id'},
                                {header: 'title',   dataIndex: 'title'},
                                {header: 'type',  dataIndex: 'type'}
                            ],
                            source:record,
                            title:record.get('title')
                        })
                     )
                });
                Ext.getCmp("accordionPanel").doLayout();
                }
            }
        }
    });
       
});

Finally only the load needs to be triggered when the button in the featureinfo panel is clicked, the id of the clicked record should be added here as a filter.

function getPanelContent(value){
store.load({params:{id:value}});
}

I hope this GXP customisation was interesting to read. Feel free to send me improvements.
I'll put up a live demo and full code-download soon.

mapwindow & metadata

Today i looked at the mapwindow/dotspatial project http://mapwindow.org. A plain-functional .Net desktop gis product. The program has a very nice plug-in architecture. Through a vast list of plug-ins quite some advanced geo-actions can be done. My interest lies in metadata, i was hoping for a CSW-search imlementation. Unfortunately it's currently not available (and no work being done), even WMS/WFS is not supported out of the box (basic WMS through a plug-in). However I was quite surprised to find a metadata-plugin in 4.8 (not in 6?). The plug-in works quite nice. A tree structure for the metadata and for every item a clear edit-form appears. The metadata is however very ESRI-FGDC oriented. In the metadataviewer are actually esri-stylesheets used.
So disappointing to find very limited support for iso19115.
I think (through the WMS-plugin) the basis is there for a CSW search plug-in, extending mapwindow in that direction would be nice. A first quick win is in the context-menu for a WMS-layer, 'show metadata' should not result in a warning ('not available') but get the metadata-link from getcapabilities and display the metadata from the WMS-server!
Maybe i'll dive in the code one day and get it done myself...
Good luck dotspatial/mapwindow team....!