My throw-away comment at the end of my earlier post shows my scepticism that the JSON file approach was really viable.
A quick ls | wc -l shows nearly one million tiles in 2024 alone. We need a different approach. There are already parquet files available, and checking register.parquet, I can see it has everything we need!
As an alternative, more scalable solution, we could have a server that loads the Parquet files using mtelvers/arrow, derived from LaurentMazare/ocaml-arrow, which can respond to queries raised by callbacks from Leaflet, allowing it to draw the required bounding boxes. Ultimately this could provide links to the Zarr data in stored in S3.
It’s a pretty simple API:
GET /years- Available yearsGET /stats?year=YYYY- Coverage statisticsGET /tiles?minx=&miny=&maxx=&maxy=&year=&limit=- Tiles in bounding boxGET /density?year=&resolution=- Tile density grid
The code is available at mtelvers/title-server and currently deployed at stac.mint.caelum.ci.dev.