Assignment 11: Spatial community composition in California
BIO597 Spatial Analysis of Biodiversity
Repeat the Lab11 analysis for California. How do grid grain, geographic distance, and climate affect our view of amphibian communities across this larger and environmentally varied state?
Use 20, 40, and 80 km cells. Follow the lab's sequence: community matrices, richness, distance decay, neighboring-community differences, GDM at each grain, and a PCA map/biplot of all 19 GDM-transformed bioclimatic predictors. You may reuse the lab's analysis code. Spatial preparation helpers are supplied below.
Submit this notebook with figures, summary tables, short answers, and a 400–600 word synthesis. Cite the GBIF download DOI and the boundary and climate sources.
Inputs and environment
Run from docs/assignments using the Lab11 environment, including gdmbayes,
GeoPandas and Rasterio.
| Input | Path from this notebook |
|---|---|
| California boundary, including islands | ../data/CA_ShapeFiles/California_State_Boundary.shp |
| GBIF amphibian occurrences | ../data/CA_amphibians.geojson |
| All 19 clipped WorldClim layers | ../data/bioclim_CA |
All inputs are provided. The occurrence GeoJSON is a reproducible random sample of California records from the GBIF download, reduced to about 20 MB and using the same fields as the Maine data. See the California data notes for filtering, sample size, and provenance. Record the download DOI here:
GBIF download DOI: 10.15468/dl.wbj54a
1. Load and inspect the California data
Use California Albers (EPSG:3310) for the grid and approximate geographic distances in meters. California spans more than one UTM zone, so do not reuse Maine's UTM projection or grid origin.
The occurrence file contains amphibian detections within the California boundary. It was randomly sampled after clipping. Drop records without a species name before constructing communities. The six fields match Maine's data: GBIF ID, species, longitude, latitude, coordinate uncertainty, and year. Random downsampling can remove observations of rarely recorded species.
from pathlib import Path
import geopandas as gpd
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import rasterio
from rasterio.mask import mask
from shapely.geometry import box
from scipy.spatial.distance import pdist, squareform
from gdmbayes import GDM
CRS = "EPSG:3310"
GRAINS = [20, 40, 80]
MIN_RECORDS = 1
CLIMATE_DIR = Path("../data/bioclim_CA")
OUTPUT = Path("outputs/Assignment11")
OUTPUT.mkdir(parents=True, exist_ok=True)
california = gpd.read_file("../data/CA_ShapeFiles/California_State_Boundary.shp")
california = california.to_crs(CRS).dissolve()
records = gpd.read_file("../data/CA_amphibians.geojson").to_crs(CRS)
print("Input records:", len(records))
records = records.dropna(subset=["species"])
records = records[records.species != ""]
records = gpd.clip(records, california)
print("Sampled records with species names:", len(records))
print("Species labels:", records.species.nunique())
# Map the boundary and occurrence points.
# Inspect record years and coordinateUncertaintyInMeters.
# Consider sampling gaps, repeated locations, and coordinate uncertainty.
Interpretation: Where are observations concentrated or sparse? Which parts of California might be especially difficult to characterize from these data? Explain why repeated records, uncertain coordinates, and unresolved species names matter for a site-by-species matrix.
Your answer:
2. Aggregate records to sites
The helper follows Lab11 but uses the California boundary and a new shared grid
origin. Finer cells nest within coarser cells. Only cells meeting MIN_RECORDS
become sites; missing cells are not treated as zero-richness communities.
Map outlines are clipped to the state, but geographic distances use the full square centers. Coastal and island cells can contain less land than interior cells of the same nominal grain.
origin_width = max(GRAINS) * 1000
x0 = np.floor(california.total_bounds[0] / origin_width) * origin_width
y0 = np.floor(california.total_bounds[1] / origin_width) * origin_width
def grid_communities(grain_km):
width = grain_km * 1000
points = records.copy()
points["col"] = ((points.geometry.x - x0) // width).astype(int)
points["row"] = ((points.geometry.y - y0) // width).astype(int)
points["site_id"] = points.col.astype(str) + "_" + points.row.astype(str)
community = pd.crosstab(points.site_id, points.species)
cell_locations = points[["site_id", "col", "row"]].drop_duplicates()
cells = []
for cell in cell_locations.itertuples():
x = x0 + cell.col * width
y = y0 + cell.row * width
cells.append({"site_id": cell.site_id, "xc": x + width / 2,
"yc": y + width / 2, "geometry": box(x, y, x + width, y + width)})
sites = gpd.GeoDataFrame(cells, crs=CRS).set_index("site_id")
sites = gpd.clip(sites, california).sort_index()
community = community.reindex(sites.index)
sites["n_records"] = community.sum(axis=1)
sites["richness"] = (community > 0).sum(axis=1)
sites = sites[sites.n_records >= MIN_RECORDS].copy()
return sites, community.loc[sites.index]
sites_by_grain = {}
communities = {}
grain_summary = pd.DataFrame()
# Loop over GRAINS and call grid_communities(grain).
# Store sites_by_grain[grain] and communities[grain], as in Lab11.
# Summarize n_sites, n_records, mean_richness, and median_records_per_site.
# Display grain_summary and the first rows of communities[40].
3. Richness and sampling maps
Plot richness for all three grains using one shared color scale. Make a second
comparison of record counts and richness at 20 km. Use california as the
background layer when adapting the lab's mapping cells.
# Create and save the three richness maps with a common color scale.
# Compare n_records and richness at 20 km.
Interpretation: Which richness patterns persist across grains? Which become more or less apparent? How much might the patterns reflect aggregation, reporting effort, or differences in the amount of land within a cell?
Your answer:
4. Distance decay of similarity
Use presence/absence and Jaccard dissimilarity. Similarity is 1 - Jaccard.
The helpers below retain matching site order and summarize similarity in
100 km distance bands, allowing for California's larger extent.
Plot all pairs and their distance-band means for each grain. Keep similarity axes fixed at 0–1. Report the descriptive Spearman distance–similarity correlation, without treating pairs that share sites as independent replicates.
def community_pairs(sites, community):
presence = community.loc[sites.index] > 0
dissimilarity = pdist(presence, metric="jaccard")
distance_km = pdist(sites[["xc", "yc"]]) / 1000
return pd.DataFrame({"distance_km": distance_km,
"jaccard": dissimilarity,
"similarity": 1 - dissimilarity})
def distance_bands(pairs):
rows = []
for lower in range(0, 1600, 100):
in_band = pairs[(pairs.distance_km >= lower) & (pairs.distance_km < lower + 100)]
if len(in_band) == 0:
continue
rows.append({"distance_km": in_band.distance_km.mean(),
"mean_similarity": in_band.similarity.mean(),
"n_pairs": len(in_band)})
return pd.DataFrame(rows)
pairs_by_grain = {}
binned_decay = {}
# For each grain, calculate community_pairs and distance_bands.
# Make distance-decay plots and calculate the descriptive correlation.
# Inspect n_pairs in the most distant bands before interpreting their means.
Interpretation: Does similarity decline steadily, level off, or show another pattern? How does the relationship change with grain? Explain at least two biological or sampling explanations. Could nested species lists contribute to Jaccard dissimilarity without species replacement?
Your answer:
5. Local differences among communities
Reuse the lab's neighbor comparison: mean Jaccard dissimilarity to sampled cells within 1.5 cell widths, excluding the focal cell. Map the results for all three grains with one 0–1 color scale. Leave cells with no sampled neighbors as missing.
# Use squareform on each grain's distance and dissimilarity vectors.
# Calculate neighbor_dissimilarity for each sampled cell.
# Map and save the three results.
Interpretation: Where are neighboring recorded communities most different? Choose two contrasting sites and inspect their species lists and record totals. Why is this not a map of pure species replacement or turnover per kilometer?
Your answer:
6. Summarize site climate
Use all 19 bioclimatic variables, matching the revised Lab11 and its predictor names. Each site's climate is the mean of valid raster pixels intersecting its land area. Edge pixels are included equally; this is an approximate zonal mean, not an exact area-weighted average.
Inspect the ranges and correlations of all predictors. For orientation, map
temperature_c (BIO1) and precip (BIO12) at 40 km. These are example maps;
all 19 variables enter every full GDM and the PCA. Several predictors describe
related aspects of climate, so their fitted contributions can overlap.
CLIMATE_LAYERS = {
'wc2.1_2.5m_bio_1.tif': 'temperature_c',
'wc2.1_2.5m_bio_2.tif': 'diurnal_range_c',
'wc2.1_2.5m_bio_3.tif': 'isothermality',
'wc2.1_2.5m_bio_4.tif': 'temp_seasonality',
'wc2.1_2.5m_bio_5.tif': 'max_temp_warmest',
'wc2.1_2.5m_bio_6.tif': 'min_temperature_coldest',
'wc2.1_2.5m_bio_7.tif': 'temp_range',
'wc2.1_2.5m_bio_8.tif': 'temp_wettest_q',
'wc2.1_2.5m_bio_9.tif': 'temp_driest_q',
'wc2.1_2.5m_bio_10.tif': 'temp_warmest_q',
'wc2.1_2.5m_bio_11.tif': 'temp_coldest_q',
'wc2.1_2.5m_bio_12.tif': 'precip',
'wc2.1_2.5m_bio_13.tif': 'precip_wettest_m',
'wc2.1_2.5m_bio_14.tif': 'precip_driest_m',
'wc2.1_2.5m_bio_15.tif': 'precip_seasonality',
'wc2.1_2.5m_bio_16.tif': 'precip_wettest_q',
'wc2.1_2.5m_bio_17.tif': 'precip_driest_q',
'wc2.1_2.5m_bio_18.tif': 'precip_warmest_q',
'wc2.1_2.5m_bio_19.tif': 'precip_coldest_q',
}
predictors = list(CLIMATE_LAYERS.values())
def add_climate(sites, climate_dir=CLIMATE_DIR):
sites = sites.copy()
for filename, name in CLIMATE_LAYERS.items():
means = []
with rasterio.open(climate_dir / filename) as raster:
raster_sites = sites.to_crs(raster.crs)
for geometry in raster_sites.geometry:
pixels, transform = mask(raster, [geometry], crop=True,
filled=False, all_touched=True)
valid_pixels = pixels[0].compressed()
if len(valid_pixels) == 0:
means.append(np.nan)
else:
means.append(valid_pixels.mean())
sites[name] = means
return sites
climate_sites = {}
# Call add_climate for each grain using all 19 layers.
# Report missing climate values and inspect all predictor ranges and correlations.
# Map temperature_c and precip at 40 km as examples of the gradients.
Interpretation: How do the environmental gradients differ from Maine's? Why might averaging climate over larger cells conceal meaningful habitat variation? What does a 1970–2000 climate average leave out when explaining occurrences collected across different years?
Your answer:
7. Fit GDM at each grain
Reuse the lab's standard GDM(geo=True, knots=0) setup. Drop sites missing
climate, align their community rows, and recompute Jaccard pairs. X must
start with xc, yc, and time_idx, followed by all 19 climate variables, in the order in predictors.
Use zero for time_idx because years are pooled; it is a required placeholder.
Fit a geography-only baseline and then geography plus climate at each grain.
Report site counts, pair counts, training deviance explained, and predictor
transformation heights. Check n_sites * (n_sites - 1) / 2 before fitting;
20/40/80 km grids keep this larger-state exercise manageable. Use all retained
sites from the supplied occurrence sample for the main comparison; do not
downsample it again.
model_sites = {}
model_X = {}
model_y = {}
models = {}
predictions = {}
gdm_summary = pd.DataFrame()
importance = pd.DataFrame()
# Prepare aligned X and y at each grain, as in Lab11.
# Fit geography-only and full GDM models, keeping model settings fixed.
# Store the full model, predictions, deviance summaries, and importance values.
# Display gdm_summary and importance.
8. Interpret the fitted models
Plot observed versus predicted Jaccard dissimilarity for each grain. Reuse the
lab's climate_curve helper to show the GDM transformations of all 19
predictors, comparing grains on each panel. A 5-by-4 panel layout leaves room
for the 19 curves. Label predictors and partial ecological distance; the x axis
uses each variable's original WorldClim scale.
Several variables are correlated. A small transformation height does not show that a climate process is biologically unimportant; another variable may represent similar information.
# Make the three observed-versus-predicted plots with 1:1 lines.
# Reuse climate_curve from Lab11 and calculate curves for all 19 predictors.
# Plot the three grains together for each predictor in a 5-by-4 panel layout.
# Label predictors and partial ecological distance; hide the unused panel.
Interpretation: How much does adding climate improve training fit over geography alone? Which environmental intervals show the most modeled change? Which conclusions persist across grains? Explain why transformation heights are not p-values, why a curve is not a habitat optimum, and why training fit is not evidence of geographic transfer accuracy.
Your answer:
9. PCA map and biplot of expected composition
Adapt the PCA cell near the end of Lab11. Start with pca_grain = 40.
Sum each predictor's weighted spline components to make 19 transformed
columns, then run PCA without standardizing them. This preserves the GDM
weights. Use the first two PC scores to color a California map and a PC1/PC2
scatterplot with the same color key. Report their explained-variance percentages.
Identify and label only the two variables with the largest loading-vector
lengths across PC1 and PC2. Calculate loadings as in the lab, then use
sqrt(PC1_loading**2 + PC2_loading**2) to measure each arrow's length. A signed
sum can cancel large loadings or change when an axis flips. Display the ranked
loadings table and draw the selected two arrows on the biplot. Also map those
two original climate variables to help interpret their spatial gradients.
Similar colors indicate similar climate-associated expected composition in the displayed two-component summary. They do not encode richness or calibrated Jaccard distance, and the remaining PCs may contain meaningful variation. GDM's separate geographic-distance effect is excluded from the site-level transformation. Each model's PCA and color scale are independent; do not compare matching colors between states or separately fitted grains as a shared key.
pca_grain = 40
# Adapt the Lab11 PCA map/biplot cell using all 19 predictors and california.
# Create a loadings table with PC1, PC2, and vector length for every predictor.
# Select the two longest vectors, display their names, and draw only their arrows.
# Save the PCA figure and map the two selected original climate variables.
Interpretation: Which two variables have the longest loading vectors? What fraction of transformed variation do the first two PCs retain? Where does modeled composition change most strongly, and how do the selected climate gradients relate to that pattern? Why might the longest PCA arrows differ from the predictors with the largest GDM transformation heights? Compare the PCA map with richness and neighbor dissimilarity. Why do these maps answer different questions?
Your answer:
11. Save results and synthesize
Save the main site layers, community matrices, distance-band summaries, GDM
inputs and predictions, model-comparison tables, all 19 climate curves, the PCA loadings table, and PCA figures
in OUTPUT. Save sensitivity results with distinct names.
Write a 400–600 word synthesis answering the main question: what can these data tell us about spatial community composition in California, and which conclusions depend on grain or sampling? Support it with numerical examples and at least three figures. Identify one concrete additional survey or biological dataset that would help distinguish competing explanations.
Your synthesis:
Sources: California data provenance and GBIF instructions, Lab11, GDM method, and gdmbayes.
# Save CSVs, GeoPackages, and figures with descriptive filenames.