Geospatial Analysis in Databricks: Enhancing Public Transit Access to Sports Facilities

Introduction

Geospatial data is experiencing massive growth. Sensor networks, mobile devices and satellites now generate vast volumes of geospatial data each day. This expansion underpins critical applications across sectors, from retail site selection and urban planning to logistics, environmental monitoring and risk analysis.

Imagine a sports complex in Flanders with no convenient access to public transport. How can urban planners ensure that such locations remain accessible? As part of a project at Aivix, we ingested three Flanders‑wide datasets (sports infrastructures, bus stops, and bus routes) and evaluated three leading spatial libraries on the Databricks platform: GeoPandas, Apache Sedona and Databricks Mosaic. Our objective was to identify the nearest bus stop for each sports facility and, for those located more than one kilometer from any stop, to determine whether a bus route passes nearby. In this post, we will explore each library’s capabilities, highlight common pitfalls and integrity checks, walk through code examples, and compare performance, usability, and cost to guide your tool selection.

GeoPandas

GeoPandas is an open‑source Python library that extends pandas for geospatial data. It can be used within Databricks notebooks, however, since it operates in a single-node environment, it is best suited for smaller datasets and local development rather than large-scale distributed processing.

Reading geospatial datasets with GeoPandas is straightforward. When a shapefile needs to be read, you just have to specify the path containing the shapefile. When another file format (e.g. a csv file) with geospatial attributes needs to be read, the typical approach is to use pandas reading functions and then convert It to a geopandas dataframe.

Geopandas supports spatial joins. For example, sjoin_nearest, a join of two GeoDataFrames based on the distance between their geometries. Here, we use it to identify sport infrastructures with its nearest bus stop.

Need for distributed Spark operations

While GeoPandas is well-suited for development and small-scale validation, it cannot leverage Spark clusters or distributed file systems, making it unsuitable for scalable production. Analysis of Flanders-wide datasets requires a distributed processing approach to ensure performance, scalability, and reliability. Spark enables parallel execution of spatial queries across large volumes of geometries. Databricks enhances this with managed clusters, automatic scaling, and robust data management through Delta Lake and Unity Catalog.

To meet these requirements, we migrated our workflows to Spark‑native operations on Databricks, first leveraging Databricks Mosaic and subsequently Apache Sedona with Photon.

Mosaic’s core H3 functions with Sedona’s spatial SQL expressions, ST_ functions, are designed to run exclusively on Photon-enabled Databricks clusters. Photon is a vectorized query engine developed by Databricks. Photon is particularly valuable for geospatial processing because its engine enables much more efficient execution of complex spatial calculations and queries. This results in faster processing of large and complex geospatial datasets, reducing turnaround times and optimizing resource usage. On top of this, some extra Maven libraries need to be installed on the cluster configuration from the init script.

Databricks Mosaic

Mosaic’s core H3 functions are based on Uber’s H3 system as can be seen on Figure 1, which provides a hierarchical hexagonal indexing method for efficiently organizing and analyzing geographic data.

Figure 1. H3 partitioning the globe into hexagons for more accurate analysis

H3 is a discrete global grid system that partitions the Earth’s surface into hexagonal cells at multiple resolutions, enabling uniform area representation and consistent neighbour relationships. Unlike square or triangular grids, hexagons in H3 ensure all neighbours are equidistant, which improves the accuracy of spatial calculations and minimizes directional bias. Originally developed by Uber for geospatial analytics in applications, H3 is now open-source and widely used for tasks such as spatial joins, heatmaps, and network modelling in both commercial and research contexts.

After adding Mosaic’s Maven coordinates to your cluster, initialization in a Databricks notebook is straightforward:

Geometries can be constructed using Mosaic functions. In this case, the st_point function was used to create point geometries for the dataframes. Unlike GeoPandas, Mosaic does not auto-detect the CRS metadata, it must be explicitly set when creating geometries.

The geometry is initially defined in EPSG:4326, which represents coordinates in degrees. To perform accurate distance-based calculations, the geometry is transformed to EPSG:31370 (Belgian Lambert 72), a projected CRS that uses meters as its unit of measurement. It is essential to retain the geometry column in EPSG:4326, as this coordinate system is required for H3 grid indexing. The indexing process converts each coordinate into a unique hexagonal cell.

This process maps each point to a corresponding H3 cell, as illustrated below:

Figure 2. Points to grid cell H3

At this stage, sport infrastructures and bus stops that fall within the same H3 cell can be joined, allowing identification of the nearest stop for each facility. Optionally, a line geometry can be constructed to visually connect the two points on a map.

A limitation of this approach is that it only considers points located within the same H3 cell. As a result, closer points in adjacent cells may be overlooked in the join, potentially leading to suboptimal matches. An issue illustrated in the figure below.

Figure 3. Lines that connect sport infrastructure with bus stop

Mosaic overcomes this by expanding each H3 cell into its six neighboring hexagons (a “k‑ring”), thereby capturing points in adjacent cells and improving nearest‑neighbor accuracy.

Figure 4. Cell based grid(2) in H3

To ensure accurate spatial joins, it is crucial to avoid common pitfalls. EPSG:4326 should be used for constructing H3 cells. However, because EPSG:4326 uses degrees rather than meters, it is not suitable for distance-based calculations. For precise distance measurements, a projected CRS such as EPSG:31370 should be employed. Additionally, when performing H3-based joins, both datasets must use the same resolution to guarantee consistency and reliable results.

Apache Sedona

Apache Sedona enhances Apache Spark by providing native support for geospatial data types and SQL functions, enabling efficient and scalable spatial analytics. It facilitates precise k-nearest neighbor (KNN) joins using spatial indexing structures such as QuadTree and STRTree. Sedona supports versatile data ingestion from a variety of formats, including Parquet, GeoParquet, shapefiles, and CSV-files containing geometries encoded in WKT or WKB. Furthermore, it offers an extensive suite of spatial predicates—such as ST_Contains, ST_Intersects, ST_Distance, and ST_Within—enabling sophisticated spatial queries and analysis.

To activate Sedona on Databricks, some jar files and libraries need to be installed via the init script on the cluster. These jar files are necessary to activate the Kryo  serializer, which speeds up the serialization and deserialization of geometry types. 

Among its key features, Sedona provides geospatial K-Nearest Neighbor join functionality for nearest-neighbor searches. In this section, we will compare Sedona’s kNN approach with the H3-based method implemented in Mosaic.

To run Apache Sedona,  you need to configure it in spark using the following command:

In Sedona, geometries are created and the SRID is assigned using the same ST_Functions  as in Mosaic.

Sedona’s spatial functions (e.g. ST_KNNDistance, ST_KNNJoin) are natively available in Spark SQL, allowing spatial operations to be written directly in SQL. This enables declarative expressions for tasks such as neighbor searches and geometry construction.

Based on tests comparing k-nearest neighbor joins using geometries in degrees versus metric units, it can be concluded that distance-based joins conducted in meters yield greater accuracy. This observation was consistent across both Sedona’s KNN join and Mosaic’s H3-based join. In both cases, using a projected CRS with meter-based units provided more reliable results for spatial proximity analyses than using geographic coordinates in degrees.

The results aligned with expectations: Mosaic’s H3-based join leverages hexagonal grid indexing and *k*-ring queries to achieve optimal performance on Databricks. However, as an approximate method, it risks missing true nearest neighbors, near grid boundaries and requires careful resolution tuning to balance precision and computational overhead. Sedona’s exact KNN join employs spatial indices (QuadTree/STRTree) to guarantee precise neighbor ordering, but incurs higher latency, memory demands, and configuration complexity (e.g., SRID assignments, JAR management), making it optimal only when strict accuracy outweighs speed and simplicity.

Comparative study

The following table gives an overview on the main characteristics of GeoPandas, Apache Sedona, and Databricks Mosaic. GeoPandas operates on a single node, designed for small datasets but struggles with large-scale workloads. Apache Sedona leverages Spark for distributed processing and C++-optimized indexing, enabling faster spatial queries and efficient handling of large data via automatic partitioning. Databricks Mosaic, tailored for enterprise-scale workloads, combines Spark with H3 indexing and Photon for linear scalability and throughput. While GeoPandas can be used on any Databricks cluster, Sedona and Mosaic requires Photon and the installation of additional libraries.

	GeoPandas	Apache Sedona	Databricks Mosaic
Scalability	Single‑node only Unsuitable for large datasets.	Fully distributed via Spark Capable of processing large-scale datasets through automatic partitioning across a cluster.	Engineered for enterprise-scale workloads within Databricks Lakehouse Harnesses Spark, H3 indexing and Photon for linear scale‑out.
Performance	Python/Shapely-based operations incur significant latency on voluminous data, though adequate for smaller tasks.	C++‑optimized indexing yields 2–10× faster spatial queries.	Photon combined with H3 indexing and Spark code generation achieves industry‑leading throughput.
Configuration	Installation via pip   Pandas-like API Reduces learning curve   Built‑in support for common spatial formats	Requires JAR-management and cluster configuration   Integrates seamlessly with Spark SQL and multiple language APIs.	Straightforward cluster installation   SQL/Scala/Python APIs; automatic registration of ST‑functions and native H3 support   Built‑in visualization

Conclusion

Our comparative study demonstrates that GeoPandas remains the go‑to solution for exploratory, small‑scale analyses, while Apache Sedona and Databricks Mosaic deliver improved performance and scalability required for enterprise large‑scale workflows. Thanks to its Photon‑powered engine and seamless H3 indexing, Mosaic emerges as the recommended default on Databricks; Sedona continues to offer precise spatial logic and broad format support when needed.

These guidelines represent just the starting point. Databricks, Mosaic and Sedona together unlock far greater possibilities, from real‑time streaming analytics to advanced spatial machine learning. As you build on this foundation, you’ll discover even more ways to harness geospatial intelligence at scale.

Written by

Hi, I’m Taha Kutlu. As an aspiring data scientist and soon-to-be graduate in Management & IT from Ghent University, I thrive at the intersection of business strategy and technology. With a background in Business Administration and a strong passion for working with complex datasets, I aim to bridge the gap between raw data and actionable insights. When I’m not coding or analyzing trends, you’ll find me hiking rugged trails, debating philosophy, or chasing a football across the field. I’m always open to new challenges and opportunities to connect!