Unable to download datasets using signed URLs

[Yeray Lopez (yeraylopez)]

Background

Following documentation for Signing a URL with a Token under Using Tokens for Data Access

Issue

When trying to download datasets BlobNotFound error is produced.

Steps to Reproduce

• Generate Signed URL with Token using
  • https://planetarycomputer.microsoft.com/api/sas/v1/sign?href=https://naipeuwest.blob.core.windows.net/naip/01.tif
• Copy signed URL under href in response
• Open URL

[Tom Augspurger (TomAugspurger)]

Are you actually using the url https://naipeuwest.blob.core.windows.net/naip/01.tif, or is that standing in for a real URL? I don't think there's an asset at that URL.

Just to be clear, you should be using URLs from the assets on the STAC items. For example, the item at https://planetarycomputer.microsoft.com/api/stac/v1/collections/naip/items/hi_m_2015561_sw_05_060_20211226_20220909 includes a link to https://naipeuwest.blob.core.windows.net/naip/v002/hi/2021/hi_060cm_2021/20155/m_2015561_sw_05_060_20211226_20220909.200.jpg.

[Yeray Lopez (yeraylopez)]

This is the actually asset I am interested in using:
https://planetarycomputer.microsoft.com/api/stac/v1/collections/us-census/items/2020-census-blocks-geo

But it seems like this asset does not have a blobstorage URL but rather a path abfs://us-census/2020/census_blocks_geo.parquet

[Tom Augspurger (TomAugspurger)]

Gotcha. abfs://us-census/2020/census_blocks_geo.parquet links to the root of a Parquet dataset, which is made up of many files. https://nbviewer.org/github/microsoft/PlanetaryComputerExamples/blob/main/tutorials/census-data.ipynb#Census-data-by-cartographic-boundaries has some information.

[Yeray Lopez (yeraylopez)]

I see that in this notebook Dask-GeoPandas is being used to read the parquet but the notebook doesn't have any code where it is saving the parquet dataset locally. Is there any documentation for that? Since that is what I am ultimately trying to do. (Ideally without using geopandas as a medium if possible)

[Tom Augspurger (TomAugspurger)]

Mmm, I'm not sure. In general, we recommend not making copies of the data locally. We recommend computing on the data in place.

If you're transforming / filtering the data, you could try something like df.to_parquet("local.parquet").

If you're just copying raw data (untransformed) I would suggest azcopy.

[Yeray Lopez (yeraylopez)]

Awesome just got azcopy to work.

Thank you so much for your support!