How to find the storage for a data collection?

[John Polo (John-Polo)]

I'm trying to access the NOAA Global Forecast System data. I think I need an API key to do this because of this note: "This dataset is not yet available in the Planetary Computer API, but can be accessed directly from Azure Blob Storage."

I read this "When a token is needed" on https://planetarycomputer.microsoft.com/docs/concepts/sas/#when-an-account-is-needed

"A SAS token is needed whenever you want to access Planetary Computer data at an Azure Blob URL."

I'm trying to use this tool, https://planetarycomputer.microsoft.com/docs/reference/sas/ to get the token. I need a storage account to use in that tool and I don't know what it is because the earlier link doesn't indicate what exactly the storage account is. I looked here
https://microsoft.github.io/AIforEarthDataSets/data/noaa-gfs.html
and I still can't tell what the storage is supposed to be. I saw this, "... East US Azure region, where the data are", but I don't know how to refer to that for the tool.

[Tom Augspurger (TomAugspurger)]

We'll need to update those docs to say "A SAS token is usually needed...." Some of our containers, notably the NOAA ones, are public. So there's no need for a SAS token in this case. Here's an example using curl:

» curl -I "https://noaagfs.blob.core.windows.net/gfs/enkfgdas.20230406/00/atmos/mem001/gdas.t00z.sfcf003.nc"
HTTP/1.1 200 OK
Content-Length: 277486090
Content-Type: application/octet-stream
Content-MD5: kf/MT8SXQT+mDxIU8nHz1g==
Last-Modified: Thu, 06 Apr 2023 07:02:41 GMT
ETag: 0x8DB366CE9F41AFA
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 98e61878-401e-0066-4a84-afae75000000
x-ms-version: 2009-09-19
x-ms-lease-status: unlocked
x-ms-blob-type: BlockBlob
Access-Control-Expose-Headers: x-ms-request-id,Server,x-ms-version,Content-Type,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
Access-Control-Allow-Origin: *
Date: Wed, 05 Jul 2023 21:02:25 GMT

and adlfs:

In [1]: import adlfs

In [2]: fs = adlfs.AzureBlobFileSystem("noaagfs")

In [3]: fs.ls("gfs")[:2]
Out[3]: ['gfs/enkfgdas.20230406', 'gfs/enkfgdas.20230407']

the earlier link doesn't indicate what exactly the storage account is.

We can spell that out more clearly, but in general, the format for Blob Storage URLs is https://<storage-account>.blob.core.windows.net/<container>. So in https://noaagfs.blob.core.windows.net/gfs the storage account is noaagfs and the container is gfs.

[John Polo (John-Polo)]

Thanks, Tom. That answers my question regarding the storage and tokens.