Serving Vector Tiles from Remote SQLite
When making custom vector map layers for use with Mapbox GL JS or Maplibre GL JS, usually that means making Mapbox Vector Tiles. There are a few options for how to create those tiles and how to serve them for use in a map. Of course there are the full services that Mapbox and Map Tiler and others offer. There are also the self hosting options, either with prerendered (using something like tippecanoe) tiles which are hosted from an object store like S3, or a tiles that are rendered at request time via PostGIS built in MVT function (and usually heavily cached) to name a couple.
For my needs I wanted a means of serving prerendered tiles that didn’t require local disk, so it can run in something like Google Cloud Run, and made use of existing tile packaged formats. There is really interesting option in PMTiles, which is a new custom format designed specifically reading tiles via http range requests. PMTiles packs tiles in such a way that max 2-3 http requests should be needed to fetch a tile. It can also be used from browser directly. It is a really good option (and the cloudflare R2/worker looks like a great setup), but I wanted to keep my tiles in MBTiles format. Its the main standard for transporting packaged Mapbox vector tiles, and can produced and read by other tools too.
MBTiles is just a SQLite file with an expected schema. So making use of ideas that I first saw in projects like sql.js-httpvfs last year, we can serve tiles from MBTiles from an object store, like Google Cloud Storage, making http range requests. Rather than doing this from a browser (too much wasm js to load and some other limitations from Mapbox GL JS) we set this up using a lightweight server that serves up an xyz tile api, and makes range requests as needed to the MBTiles object.
There are a couple small optimizations worth mentioning. First, SQLite reads from the file in pages, which by default are 4kb. 4kb can be really small and result in a lot of requests when it comes to tiles that might be 100-500kb each. So when a page is requested, rather than make an http range request for only 4kb, we read extra pages and cache it for the next SQLite read. We cache those in a LRU so depending on where this is being run, that cache can greatly reduce the number of http range requests. Second, an index for the xyz tilename isn’t required by the mbtile spec, but is very important especially when reading MBTiles from a remote object store.
My main workflow for vector tiles these days is creating them in Big Query (more on that in the future), exporting to MBTiles on GCS, and serving them via Cloud Run. If folks are interested I intend make more of that workflow open source, but as a start the tileserver that serves tiles from MBTiles using range requests from an object store is now on GitHub.
An example of it serving some demo vector tiles (USA States)
If you end up making use of this or have questions please let me know, either via mick at mick dot im or in the issues on github.