Python Geospatial Analysis Essentials

By Erik Westra

About This Book

• Learn to build a complete geospatial application from scratch using Python
• Create good-looking maps based on the results of your analysis
• This is a fast-paced guide to help you explore the key concepts of geospatial to obtain high quality spatial data

Who This Book Is For

If you are an experienced Python developer and wish to get up-to-speed with geospatial programming, then this book is for you. While familiarity with installing third-party Python libraries would be an advantage, no prior knowledge of geospatial programming is required.

• Language: English
• Publisher: Packt Publishing
• Release date: Jun 23, 2015
• ISBN: 9781783553891

Erik Westra

Erik Westra has been a professional software developer for over 25 years, and has worked almost exclusively in Python for the past decade. Erik's early interest in graphical user-interface design led to the development of one of the most advanced urgent courier dispatch systems used by messenger and courier companies worldwide.In recent years, Erik has been involved in the design and implementation of systems matching seeks and providers of goods and services across a range of geographical areas. This work has included the creation of real-time geocoders and map-based views of constantly changing data. Erik is based in New Zealand, and works for companies worldwide.

Book preview

Table of Contents

Python Geospatial Analysis Essentials

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Geospatial Analysis and Techniques

About geospatial analysis

Understanding geospatial data

Setting up your Python installation

Installing GDAL

Installing Shapely

Obtaining some geospatial data

Unlocking the shapefile

Analyzing the data

A program to identify neighboring countries

Summary

2. Geospatial Data

Geospatial data quality

Types of geospatial data

Shapefiles

Well-known text

Well-known binary

Spatial databases

Geospatial microformats

GeoJSON

GML

Digital elevation models

Raster basemaps

Multiband raster files

Sources of freely available geospatial data

Natural Earth Data

OpenStreetMap

US Census Bureau

World Borders Dataset

GLOBE

National Elevation Dataset

Reading and writing geospatial data using Python

Reading vector data

Writing vector data

Reading raster data

Writing raster data

Dealing with spatial reference systems

WGS84

Universal Transverse Mercator

Describing spatial reference systems

Transforming coordinates

Calculating lengths and areas

Geospatial data errors and how to fix them

Points

LineStrings

Linear Rings

Polygons

MultiPolygons

Fixing invalid geometries

Summary

3. Spatial Databases

Spatial database concepts

Installing a spatial database

Installing PostgreSQL

Installing PostGIS

Installing psycopg2

Accessing PostGIS from Python

Setting up a spatial database

Importing spatial data

Querying spatial data

Manipulating spatial data

Exporting spatial data

Summary

4. Creating Maps

Introducing Mapnik

Installing Mapnik

A taste of Mapnik

Building a map

Styling a map

Learning Mapnik

Datasources

Symbolizers

PointSymbolizer

LineSymbolizer

PolygonSymbolizer

TextSymbolizer

RasterSymbolizer

Map rendering

A working example

Next steps

Summary

5. Analyzing Geospatial Data

Libraries for spatial analysis

PyProj

NetworkX

Spatial analysis recipes

Calculating and comparing coordinates

Calculating lengths

Calculating areas

Calculating shortest paths

Summary

6. Building a Complete Geospatial Analysis System

Matching GPS data against a map

An overview of the GPS Heatmap system

Obtaining the necessary data

Obtaining GPS data

Downloading the road data

Implementing the GPS Heatmap system

Initializing the database

Importing the road data

Splitting the road data into segments

Constructing a network of directed road segments

Implementing the map matching algorithm

Generating the GPS heatmap

Further improvements

Summary

Index

Python Geospatial Analysis Essentials

---

Python Geospatial Analysis Essentials

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: June 2015

Production reference: 1150615

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78217-451-6

www.packtpub.com

Credits

Author

Erik Westra

Reviewers

Min Feng

Eric Hardin

Richard Marsden

Puneet Narula

Ryan Small

Commissioning Editor

Amarabha Banerjee

Acquisition Editor

Vinay Argekar

Content Development Editor

Mamata Walkar

Technical Editor

Pankaj Kadam

Copy Editors

Puja Lalwani

Laxmi Subramanian

Project Coordinator

Shipra Chawhan

Proofreaders

Stephen Copestake

Safis Editing

Indexer

Tejal Soni

Production Coordinator

Manu Joseph

Cover Work

Manu Joseph

About the Author

Erik Westra has been a professional software developer for over 25 years now, and he has worked almost exclusively in Python for the past decade. Erik's early interest in graphical user interface design led to the development of one of the most advanced urgent courier dispatch systems used by messenger and courier companies worldwide.

In recent years, Erik has been involved in the design and implementation of systems matching seekers and providers of goods and services across a range of geographical areas, as well as real-time messaging and payment systems. This work has included the creation of real-time geocoders and map-based views of constantly changing data. Erik is based in New Zealand, and works for companies worldwide.

Erik is also the author of the titles Python Geospatial Development and Building Mapping Applications with QGIS, both by Packt Publishing.

About the Reviewers

Min Feng completed his PhD degree in cartography and geographic information systems in 2008 and has been working at the Global Land Cover Facility (GLCF), University of Maryland, Chinese Academy of Sciences (CAS), and Global Resource Information Database (GRID) of the United Nations Environment Programme (UNEP). Dr. Feng has been engaged in global high-resolution land cover and change research and is an expert with high-performance geospatial data processing, geospatial model sharing, and integrated simulation. His work has been published in top-ranking remote sensing and GIS journals. Dr. Feng is familiar with OGC/ISO standards and open source tools and libraries, and is also capable of programming using many languages, including C/C++, Java, Python, R, and IDL. He has also reviewed Learning QGIS – Second Edition, Packt Publishing.

Eric Hardin works in the defense and aerospace industry as a software engineer with an international research and engineering company. While pursuing a PhD in physics, he used geospatial analysis and process-based modeling to study coastal geomorphology and develop robust storm hazard mapping techniques. In his current role, he has supported the development of weaponeering simulation software as well as software designed to automate the processing of LiDAR and other geospatial data in near-real time. Although opportunities are not as prevalent as in past years, he still gets excited about all things geospatial and finds any excuse to use Python—even when it's a little overkill.

Richard Marsden has over 15 years of professional software development experience. After starting in the fields of geophysics and oil exploration, he has spent the last 12 years running the Winwaed Software Technology LLC independent software vendor. Winwaed specializes in geospatial tools and applications, including web applications, and operates the http://www.mapping-tools.com website for tools and add-ins for geospatial products, such as Caliper Maptitude and Microsoft MapPoint.

Richard was also a technical reviewer on Python Geospatial Development, Packt Publishing.

Puneet Narula is currently working as a PPC data analyst with Hostelworld.com Limited (www.hostelworld.com), Dublin, Ireland, where he analyzes masses of clickstream data from both direct and affiliate sources and provides insights for the digital marketing team. He uses RapidMiner, R, and Python for exploratory and predictive analysis. His areas of expertise are programming in Python and R, machine learning, data analysis, and visualization.

He started his career in banking and finance and then moved to the ever-growing domain of data and analytics.

He earned an MSc in computing (data analytics) from Dublin Institute of Technology, Dublin, Ireland. He has also reviewed Python Data Analysis, Packt Publishing.

Ryan Small is a technology generalist based out of Seattle, Washington. He is an active participant in the Cascadia Chapter of OSgeo and a DevOps engineer for ClipCard Inc. Ryan has a passion for solving software and infrastructure problems, especially those with a geospatial angle.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

There are several powerful Python libraries for reading, processing, analyzing, and viewing geospatial data. There are also a number of websites that provide high-quality geospatial data, which you can use freely in your own projects. This data will often be the basis for your analysis, providing the shapes of countries, the positions of cities, the outlines of roads, and so on. Using this data in conjunction with the available geospatial libraries gives you a powerful toolkit for performing your own geospatial analysis using Python.

What this book covers

Chapter 1, Geospatial Analysis and Techniques, walks the reader through the process of downloading sample geospatial data, before writing a simple Python program to read and analyze that sample data.

Chapter 2, Geospatial Data, focuses on the data used for geospatial analysis: how to obtain it, why good data is important, the different formats that geospatial data can come in, and how to generate your own spatial datasets.

Chapter 3, Spatial Databases, provides a brief introduction to creating geospatial databases, how to store data in a spatially-enabled database, and how to perform efficient queries against that data.

Chapter 4, Creating Maps, looks at how to use the Mapnik library to produce great-looking maps.

Chapter 5, Analyzing Geospatial Data, guides the reader through the process of writing spatial analysis programs using Python. Based on the datasets downloaded in Chapter 2, Geospatial Data, and using the major Python libraries for geospatial analysis, this chapter uses a recipe-like format to solve a range of typical spatial analysis problems.

Chapter 6, Building a Complete Geospatial Analysis System, uses all the various libraries and techniques covered in the earlier chapters to build a complete geospatial analysis system.

What you need for this book

The code examples in this book use Python 2 to analyze geospatial data. Any reasonably powerful computer running Windows, Mac OS X, or Linux will be suitable. You will need to download and install the following software onto your computer:

Python version 2.7 or later, excluding Python 3.x

GDAL/OGR version 1.11 or later

GEOS version 3.4.2 or later

Shapely version 1.5.7 or later

PostgreSQL version 9.3 or later

PostGIS version 2.1.4 or later

psycopg2 version 2.5 or later

Mapnik version 2.2 or later

PROJ version 4.0 or later

PyProj version 1.9.4 or later

NetworkX version 1.9.1 or later

Full instructions for downloading, installing, and using these various tools and libraries are included in this book.

Who this book is for

If you are an experienced Python developer wishing to come up to speed with geospatial programming, or have specific spatial programming needs, then this book is for you. While familiarity with installing third-party Python libraries will be an advantage, no prior knowledge of geospatial programming concepts or techniques is required.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: Once you have installed it, you can check that it's working by firing up your Python interpreter and typing import osgeo.gdal and then import osgeo.ogr.

A block of code is set as follows:

import osgeo.ogr

shapefile = osgeo.ogr.Open(TM_WORLD_BORDERS-0.3.shp)

layer = shapefile.GetLayer(0)

for i in range(layer.GetFeatureCount()):

    feature = layer.GetFeature(i)

    feature_name = feature.GetField(NAME)

    geometry = feature.GetGeometryRef()

    geometry_type = geometry.GetGeometryName()

    print i, feature_name, geometry_type

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

from osgeo import ogr

driver = ogr.GetDriverByName(ESRI Shapefile)

dstFile = driver.

CreateDataSource(test-shapefile)

Any command-line input or output is written as follows:

% python readRaster.py -500 53081919 -84 1 -83 8 -82 9 -81 17 ... 5241 1 5295 1 5300 1 5443 1

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: Clicking on the Next button moves you to the next screen.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[e-mail]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and