Lab 04: Vector Spatial Analysis
BIO597 Spatial Analysis of Biodiversity
This lab uses Maine amphibian occurrence records together with conserved lands, HUC12 watersheds, and major roads. Each section starts with a biological or conservation question and introduces one spatial operation that can help answer it.
Goals
By the end of this lab, you should be able to:
- explain why vector layers need a coordinate reference system
- use a spatial join to connect points with polygons
- create a buffer using a distance measured in meters
- summarize occurrence records by watershed
- find the nearest road to each occurrence
- use intersection and difference to compare polygon layers
The vector layers
We will use four kinds of vector data:
- points: amphibian occurrence records
- polygons: conserved lands
- polygons: HUC12 watersheds
- lines: primary and secondary roads
The shapefiles are in ME_ShapeFiles. See the README in that directory for the source and limitations of each layer. The road layer contains major roads, not every local road.
1. Import packages
pandas reads the occurrence table. geopandas reads and analyzes spatial data.
import pandas as pd
import geopandas as gpd
2. Load the occurrence records
This is the same tab-delimited GBIF file used in Assignment 03. We will keep only the columns needed for today's spatial questions.
amphibians = pd.read_csv(
"../assignments/Amphibians_ME/Amphibians_ME.csv",
sep="\t",
low_memory=False,
)
amphibians.shape
columns_to_keep = [
"gbifID",
"species",
"decimalLongitude",
"decimalLatitude",
"coordinateUncertaintyInMeters",
"year",
]
amphibians = amphibians[columns_to_keep]
amphibians.head()
3. Prepare the coordinates
Spatial analysis requires usable numeric coordinates. Convert longitude and latitude to numbers, then remove records that are missing a species name or either coordinate.
amphibians["decimalLongitude"] = pd.to_numeric(
amphibians["decimalLongitude"], errors="coerce"
)
amphibians["decimalLatitude"] = pd.to_numeric(
amphibians["decimalLatitude"], errors="coerce"
)
amphibians = amphibians.dropna(
subset=["species", "decimalLongitude", "decimalLatitude"]
).copy()
amphibians.shape
4. Create a GeoDataFrame
GBIF coordinates use longitude and latitude. EPSG:4326 describes that coordinate reference system.
amphibians_gdf = gpd.GeoDataFrame(
amphibians,
geometry=gpd.points_from_xy(
amphibians["decimalLongitude"],
amphibians["decimalLatitude"],
),
crs="EPSG:4326",
)
amphibians_gdf.head()
5. Load the Maine vector layers
gpd.read_file() can read a shapefile directly. Remember that the other files beside each .shp file are also parts of the shapefile and must stay together.
conserved = gpd.read_file("ME_ShapeFiles/Maine_Conserved_Lands.shp")
watersheds = gpd.read_file("ME_ShapeFiles/Maine_HUC12_Watersheds.shp")
roads = gpd.read_file("ME_ShapeFiles/Maine_Primary_Secondary_Roads.shp")
6. Inspect geometry types and coordinate systems
The geometry type tells us what each row represents. The CRS tells us how coordinates are stored.
print("Occurrences:", amphibians_gdf.geometry.geom_type.unique(), amphibians_gdf.crs)
print("Conserved lands:", conserved.geometry.geom_type.unique(), conserved.crs)
print("Watersheds:", watersheds.geometry.geom_type.unique(), watersheds.crs)
print("Roads:", roads.geometry.geom_type.unique(), roads.crs)
7. Make quick maps
First inspect each layer separately. Interactive maps are useful for checking whether a layer is in the expected location.
conserved.explore(tooltip=["PARCEL_NAM", "CONS1_TYPE", "PUB_ACCESS"])
watersheds.explore(tooltip=["huc12", "name"])
roads.explore(tooltip=["FULLNAME", "MTFCC"])
8. Project the layers before measuring distance
Longitude and latitude are measured in degrees. Buffers and distances are easier to interpret in a projected CRS whose units are meters.
We will use EPSG:26919, UTM Zone 19N, for Maine. Every layer used in the same spatial operation must use the same CRS.
maine_crs = "EPSG:26919"
amphibians_projected = amphibians_gdf.to_crs(maine_crs)
conserved_projected = conserved.to_crs(maine_crs)
watersheds_projected = watersheds.to_crs(maine_crs)
roads_projected = roads.to_crs(maine_crs)
print(amphibians_projected.crs)
print(conserved_projected.crs)
print(watersheds_projected.crs)
print(roads_projected.crs)
Question 1: Which amphibian records occur inside conserved lands?
A spatial join adds attributes from one spatial layer to another according to a spatial relationship. Here, an occurrence point matches a conserved-land polygon when the point is within that polygon.
We use an inner join because we only want matching occurrence records.
conserved_columns = ["PARCEL_NAM", "CONS1_TYPE", "PUB_ACCESS", "geometry"]
occurrences_in_conserved = gpd.sjoin(
amphibians_projected,
conserved_projected[conserved_columns],
how="inner",
predicate="within",
)
print(amphibians_projected.shape, occurrences_in_conserved.shape)
occurrences_in_conserved.head()
One occurrence can match more than one overlapping conservation polygon. Count unique gbifID values when the question is about occurrence records.
unique_conserved_records = occurrences_in_conserved.drop_duplicates(subset="gbifID")
len(unique_conserved_records)
conserved_species_counts = unique_conserved_records["species"].value_counts()
conserved_species_counts
Map the matching points. Convert them back to longitude and latitude so the interactive basemap aligns correctly.
unique_conserved_records.to_crs("EPSG:4326").explore(
column="species",
tooltip=["species", "PARCEL_NAM", "PUB_ACCESS"],
)
Question 2: Which watershed contains each occurrence?
HUC12 polygons represent small drainage areas. A point-in-polygon spatial join can assign each occurrence to a watershed.
watershed_columns = ["huc12", "name", "geometry"]
occurrences_by_watershed = gpd.sjoin(
amphibians_projected,
watersheds_projected[watershed_columns],
how="inner",
predicate="within",
)
occurrences_by_watershed.head()
Now change the unit of analysis from individual occurrence records to watersheds. For each watershed, count unique records and unique species.
watershed_summary = occurrences_by_watershed.groupby(
["huc12", "name"]
).agg(
records=("gbifID", "nunique"),
species_count=("species", "nunique"),
)
watershed_summary = watershed_summary.reset_index()
watershed_summary.head()
watershed_summary.sort_values("species_count", ascending=False).head(10)
Join the summary back to the watershed polygons so the counts can be mapped.
watersheds_with_counts = watersheds_projected.merge(
watershed_summary,
on=["huc12", "name"],
how="left",
)
watersheds_with_counts.to_crs("EPSG:4326").explore(
column="species_count",
tooltip=["name", "huc12", "records", "species_count"],
)
Question 3: How many species have records within 500 m of a major road?
A buffer creates a polygon covering all locations within a chosen distance of a feature. Because these roads are projected in meters, a buffer distance of 500 means 500 meters.
road_buffer_geometry = roads_projected.buffer(500)
road_buffer_geometry.head()
The individual road buffers overlap. union_all() combines them into one geometry so an occurrence is not counted repeatedly for several nearby road segments.
road_buffer_union = road_buffer_geometry.union_all()
road_buffer = gpd.GeoDataFrame(
{"buffer_m": [500]},
geometry=[road_buffer_union],
crs=maine_crs,
)
occurrences_near_roads = gpd.sjoin(
amphibians_projected,
road_buffer,
how="inner",
predicate="within",
)
occurrences_near_roads.shape
species_near_roads = occurrences_near_roads["species"].nunique()
species_near_roads
occurrences_near_roads["species"].value_counts()
This result is about proximity to primary and secondary roads. It does not measure proximity to every local road, and it does not show that roads caused the observed pattern.
Question 4: How far is each occurrence from the nearest major road?
sjoin_nearest() finds the closest feature in another layer. The new distance_to_road_m column records the distance because our projected CRS uses meters.
road_columns = ["FULLNAME", "MTFCC", "geometry"]
nearest_roads = gpd.sjoin_nearest(
amphibians_projected,
roads_projected[road_columns],
how="left",
distance_col="distance_to_road_m",
)
nearest_roads.head()
nearest_roads["distance_to_road_m"].describe()
The nearest-road distances form a one-column distance matrix: every occurrence has a distance to its closest road. A complete distance matrix would compare every occurrence with every road or every other occurrence, which would be much larger.
Polygon overlay: intersection and difference
An intersection keeps places shared by two layers. A difference keeps places in the first layer that are not covered by the second.
To keep this example small, use the watershed with the most occurrence records.
top_watershed = watershed_summary.sort_values(
"records", ascending=False
).iloc[0]
top_huc12 = top_watershed["huc12"]
top_huc12
study_watershed = watersheds_projected[
watersheds_projected["huc12"] == top_huc12
].copy()
study_watershed[["huc12", "name"]]
First clip the conserved lands to the watershed using intersection.
conserved_in_watershed = gpd.overlay(
conserved_projected,
study_watershed,
how="intersection",
)
conserved_in_watershed.shape
dissolve() demonstrates a union-like operation by combining the intersecting conserved parcels. We then use difference to retain the part of the watershed outside those conserved parcels.
conserved_union = conserved_in_watershed.dissolve()
unconserved_part = gpd.overlay(
study_watershed,
conserved_union,
how="difference",
)
m = study_watershed.explore()
unconserved_part.explore(m=m)
conserved_area_km2 = conserved_in_watershed.area.sum() / 1_000_000
watershed_area_km2 = study_watershed.area.sum() / 1_000_000
100 * conserved_area_km2 / watershed_area_km2
The percentage above is a geometric estimate based on the supplied layers. Conserved lands are approximate planning boundaries, and this calculation should not be treated as a legal acreage determination.
Discussion questions
- How did the unit of analysis change when we summarized occurrence records by watershed?
- Why was a projected CRS required for the 500 m buffer and nearest-road distance?
- Why might occurrence records be especially common near roads even if roads are not good amphibian habitat?
- What information is lost when overlapping road buffers are combined with
union_all()? - Which result in this notebook is most sensitive to sampling bias?
Practice submitting your work to GitHub
Run the notebook from top to bottom. Save it, then add, commit, and push the notebook from a terminal.
git add docs/labs/Lab04-VectorSpatialAnalysis.ipynb
git commit -m "Complete vector spatial analysis lab"
git push