Advertisements

Posts Tagged ‘sig’

Risk exposure. Geoprocessing using Open Source Data!! Next steps!!

2016/01/29

Now that we have completed a first example, let’s continue with a real-world one. Its important working on a Data Model to define what we understand as a Risk and how important this is. Meaning. High voltage power lines are an actual risk but the closer we are, i guess the bigger the risk is, meaning i.e 3 if we are within 50m and 1 if we are 150m away… It’s only a guess.

risk3

Same thing related to antennas, Petrol stations, etc.

This is my Data Model defined over the city of Madrid, Spain.

1 LINES- Roads speed >50 km/h within 100m risk=3
2 LINES- Power lines within 100m risk=3

3 POINTS-

Antenna,
High voltage towers,
Petrol stations:

risk if within 50m=3; risk if within 100m=2; risk if within 150m=1; 

4 AREAS-

Cement factories,
Electric Sub-stations,
Waste storage facilities:

risk if within 50m=3; risk if within 100m=2; risk if within 150m=1; 

(NOTE: You can choose your own risk thresholds and importance. Also note these information downloaded from Open Source data (Cartociudad, CNIG) has not been double checked and it has been used as is).

risk2risk1

How is this risk, or these combination of risks impacting in the population of Madrid?

risk4

Can we extrapolate these patterns to other cities in the world?
We will definitely continue  this analysis shortly.

You can also visuallize this analysis using CartoDB, the field regarding “risk exposure level” is called ALL2, and ranges from 2 to 12:

Software: ArcGIS 10.3, Global Mapper 17, CartoDB

Please share if you enjoyed it… or just to say hello!

Alberto C
MSc GIS and remote sensing UAH

Advertisements

Risk exposure. First steps

2016/01/15

Knowing how to geoprocess features is key if what we want is assesing risk exposure. What’s a risk? Which are the risks? Where are the risks? How important a risk is?

risk-exposure-fist-steps-20160115

Jugando con CartoDB

2016/01/15

Hace ya mucho tiempo que he oído hablar de CartoDB y que vengo practicando en su página web a visualizar bases de datos sencillas.

  1. Crea una cuenta
  2. Incorpora tus datos o tómalos de la galería
  3. Selecciona en modo datos la columna que quieres simbolizar/visualizar
  4. Conviértela en NUMBER si estuviera en STRING
  5. Selecciona en modo mapa en WIZARD
  6. COROPLETAS, columna _población
  7. Visualiza el resultado

cartodb03cartodb01cartodb02

Change detection – Detección de cambios en polígonos

2015/10/22

change-detection-bogota-telemediciones-20151023-02
THE IDEA: DEMONSTRATING HOW DYNAMIC A CITY IS, THUS HOW IMPORTANT IS HAVING AN UPDATED DATASET
bogota-change-detection-20151105-02

THE FACTS: THE CITY OF BOGOTÁ IN COLOMBIA 2012-2014

Overall growth rate: -0.12% ONLY HAVING INTO ACCOUNT THE DIFFERENCE OF BUILDINGS CAPTURED BETWEEN 2012 AND 2014 (We can do this because we have used the same data capture model in both years)

(De acuerdo al censo catastral, para 2015 la ciudad incorporó 51.531 predios nuevos urbanos. En total, hay 2’402.581 predios en la ciudad, de esos, 266,9 millones de metros cuadrados son de área totalmente edificada. Source: http://www.eltiempo.com/bogota/crecimiento-bogota-/15394797)

bogota-change-detection-20151105

THE PROCEDURE: Centroids of buildings; Spatial join showing presence-absence, considering a 10m accuracy threshold, meaning if the centroid has not moved more than 10m, its the same building. If the centroid in 2012 is not in 2014, its considered as demolished. If a new centroid appears its considered new building.

DENSITY MAPS+3D buildings
Help to quickly focus on the highlights
bogota-change-detection-news-20151021

 

La geográfica cabecera de ‘Up in the air’ con George Clooney

2015/08/24

Para alguien que maneja mapas a diario es una sorpresa interesante cruzarse con esta cabecera de la película de Jason Reirman protagonizada por George Clooney ‘Up in the air’. Nubes, campos de cultivo, ciudades en 2D, 3D. Qué bien he elegido mi profesión:-)

Para alguien que viaja mucho, salvando las distancias, este vídeo, también del mismo fil ‘Up in the air’ es un simpático recuerdo de lo que es un viaje pasando por uno y mil auropuertos, arcos voltaicos, empaquetando, desempaquetando…

Espero que os guste!

Alberto

(Fuente: Jose Ignacio Sánchez de Nosolosig)

DTM validation using Google Earth (and RMSE extraction)

2015/03/10

Hi guys,

Surfing the internet is great when you need to figure out something. I needed to validate some DTM from unknown sources against an also unknown source (but at least a kind of reliable one, Google Earth).

All we need is

  • Google Earth
  • TCX converter
  • ARcGIS
  • Excel

This is the procedure i have followed:

  1. First of all we draw a path over our AOI using Google Earth, we save this as KML,
  2. This KML is opened by TCX converter, added heights and exported as CSV,
  3. CSV is imported by ArcGIS,
  4. We use the tool ‘extract multi values to points‘ to get in the same table the values of our DTM and the values from Google Earth,
  5. We use Excel to calculate the RMSE and get a quantitative result,

These are the values in our DTM

dtm-validation-02

This is the path we have to draw in Google Earth

dtm-validation-03

Using TCX converter we get the heights out of Google Earth’s DTM

dtm-validation-01

Using the tool ‘extract multi values to points‘ we get the heights out of our DTM

dtm-validation-04

We measure the differences and extract the RMSE.
Are we within our acceptance threshold or expected level of accuracy?.

You guys have to figure this out for yourselves!!!

Lost regarding RMSE calculation?. Think you have to take a look at this other post.

dtm-validation-05

dtm-validation-06

Hope you guys have enjoyed this post, if so, don’t forget sharing it.

Alberto Concejal
MSc GIS and QCQA expert (well this is my post and i say what i want :-))

Pearson correlation and GIS

2014/11/28


pearson-01
Do these two variables have a correlation?. To answer this important question first of all we have to know that only if it’s a linear relationship and there are no outliers we can take advantage of Mr Pearson’s correlation statiscal tool.

If i love chocolate, does this mean i have tendency of being chuby? or on the other hand there’s no relationship at all. Let’s figure it out.

For this particular occasion, input data XY are two DTM heights, my guess is the following: if correlation is too big, i may deduce they’re not independent products and one might been created from the other, in other words, we might have tried to cheat and we are using a different source that the one we have stated… In GIS sometimes things are not exactly as expected and there’s need to be assertive and making a plan for discovering this minor issues.

 

 

 

Let’s start from the beginning, if source 1 is the same as source 2, the correlation would be perfect, is this correct?. The answer is yes. r (Person correlation) would be = 1. So yes, if this was asking about chocolate and fleshiness this would be 100% right but this hardly or never happens in real life (direct and no other explanation or variable interaction… why is always so0o complicated?).

pearson-formula

pearson-04

With real data, you would not expect to get values of r of exactly -1, 0, or 1. For example, the data for spousal ages (white couples) has an r of 0.97. Don’t ask me where i got this weird source (well, just in case: http://onlinestatbook.com/2/describing_bivariate_data/intro.html)

age_scatterplot

If i fill source 2 with a random number, the correlation would be almost none accordingly (in this case r=0.17)

pearson-06

Now if we see the diagram of the first two sources and we get the Pearson correlation coefficient (r=0.24) which means the correlation is very weak.

pearson-03

But that was only a very small part of the table (only 30 iterations), so if i do the same calculation out of the +13,000 iterations i really need, i get these figures (by the way, theres no need to use such a complicated formula above, you can use this one in EXCEL: =PEARSON(A1:An;B1:Bn))

pearson-07

So the correlation now its moderate, which makes me deduct at least the sources seem different and i’d need more clues to think my customer might have tried to actually cheat me using the same source for both datasets.

Summarizing:

r=1, correlation is PERFECT

0.75<r<1, correlation is STRONG

0.5<r<0.75, correlation is MODERATE

0.25<r<0.5, correlation is WEAK

<0.25, almost NO correlation, both variables are hardy related

I hope you guys have found this post interesting,
looking forward to hear where could you use it and/or your feedback,

Regards,

Alberto Concejal
MSc GIS

‘Reality Checks’, also called ‘Ground Truth Tests’

2014/08/19

Comparing all kind of Geodata (i.e 3D Buildings, DTM, DHM, DSM, Land Use, vectors,…) to background sources as Google Earth/ Bing, available sources from the country we are working on or WMS available sources, etc.

reality-checks

Figuring out if the data requested and we want to deliver is consistent enough compared to the so called “Truth”. Some of these checks are visual/manual, some others are more automated/analytic, Preparing ad-hoc reports using Photoshop macros to explain/flag/highlight etc. also videos, PPTs, specific ‘White Papers’ and any other way of facilitating the comprehension of the potential issues.

RSME comparing LIDAR data with a third party’s 3D dataset

2014/05/02

I would like to share with you an easy analysis i have been working in the last days. I had a vector dataset of buildings and i knew how high they were (there was a field called ‘AGL’ or Above Ground Level) and a LIDAR 2m resolution dataset over the city of London. My aim was comparing both sources, understanding LIDAR data was the actual reality (or a closer version to it) and my source of 3D buildings was the dataset i needed to deliver to my customer…  Te actual height of those 3D buildings had been extracted using stereo photogrammetry methods. I also needed to focus on residential data, so heights below 15m… So make it easy. The question was:

How accurate is my dataset of residential buildings over London?. Which is the RMSE measuring them both?

I used Global Mapper v.13.2 (b062012) and ArcGIS 10.0 (b3200)

This is the 2m resolution LIDAR data provided by geomatics-group.co.uk

LIDAR-01

I also needed to get a layer of points out of this dataset so i used Global Mapper and went to Files/Export elevation grid format and choose ASCII as the format.LIDAR-06

This is the layer of buildings and their AGL as label
LIDAR-02

I flagged those residential buildings
LIDAR-03

and using ArcGIS i performed a Spatial Analysis using Arctoolbox/Spatial analysis to join the Lidar heights in ASCII format and the residential heights… to be able to measure the difference between both datasets

this way i got a new vector layer which table contained both elevation fields (Lidar and my 3D buildings)
LIDAR-07

As you can see, i added a new field in ArcGis using table/add field and added ‘compare’ and SQL [“AGL”- “ELEVATION”]
LIDAR-04

then i measured it visually using a density grid in Global Mapper. Create density Grid.
LIDAR-05

And finally measured the RMSE by opening the table in excell format and usign the actual formula for extracting RSME values:

= SQRT(SUMSQ(M1:Mn)/COUNTA(M1:Mn)) —> Note this formula is only valid for this case. You’d need to update Mx values using yours:-)

LIDAR-07

Wow! a very high value. Does this value corresponds to our accuracy figures? Yes? No?.

Now it’s the time for decission makers to bring into action!

LIDAR-08

And what about some geostatistical analysis. I performed this using North East Trends in ArcGis. We can see from West to East there’s no variation  but we can see it increases the error the further the south…

LIDAR-09

So this is the area concentrating the higher differences comparing both datasets.

Hope you liked the analysis, if so…share!!!!

LIDAR-10

Alberto CONCEJAL
MSc GIS

 

PISA report SPAIN 2012 by Comunidades Autónomas Trends North-South and West-East

2013/12/10

I wanted to take advantage of the latest ‘advanced course on Statistical Analysis’ followed in the spanish IGN and show you how North-South and West-East trends impact on this report.
As you can see, the higher the values, the most likely to be in the North and the East of the country. This doesn’t mean we are not going to find high values in other CCAA, but this is the result of the trend.

What’s a Comunidad Autónoma? (the administrative level used for this report), You can read it here.

Let say for instance we want to focus on Reading Literacy. The highest values trend on being over the Northern and Eastern Comunidades Autonomas: Navarra, but we also find high values over La Rioja, Asturias, Euskadi or Aragón (they’re all in the North).

pisa-report-by-ccaa-20131210-02

The blue line shows how figures are increasing if we move to the North, likewise, if we look at the green line, higher values are in the eastern side of the Map.

pisa-report-by-ccaa-20131210

Also note how the lowest values fall over the Comunidades Autónomas located in the South and the West of Spain (I.e Andalucía and Extremadura).
It’s also important highlighting there are some outliers or regions with low values in the West (Murcia) or in the North but this is all about Trends!!! : -)

Source: WIKIPEDIA (http://es.wikipedia.org/wiki/Informe_PISA) and own elaboration
Software used: ArcGIS 10.0 /Geostatistical Analyst extension
20131220

geostatistical-analyst

I hope you guys find it interesting, please dont hesitate to ask if i can explain you any further.
Thank you for your time,

Alberto CONCEJAL
MSc GIS