## Stage 1 pick a sample local era5.nc and tokenize to hp
##### Stage 1 pick a sample local era5.nc and tokenize to hp
1 - Assuming rectilinear and plot a sample
1 - Assuming rectilinear and plot a sample
### 2 - Assuming rectilinear and plot across variables and samples
2 - Assuming rectilinear and plot across variables and samples
### 3 - save the hp without loosing data
3 - save the hp without loosing data
### 4 - try handling native spectral guassian thats reduced than full rectilinear as in ERA5 with no overlap
4 - try handling native spectral guassian thats reduced than full rectilinear as in ERA5 with no overlap
### 5 - same as 4 but with overlap
5 - same as 4 but with overlap
## Stage 2 Fetch data directly from CDS API of ERA5 and convert to healpix
##### Stage 2 Fetch data directly from CDS API of ERA5 and convert to healpix
### Without loss of data (Overlapping no considered for metadata at this point) - config is handled by hydra and saved as different formats as needed tested end to end with netcdf
- Without loss of data (Overlapping no considered for metadata at this point) - config is handled by hydra and saved as different formats as needed tested end to end with netcdf
## Stage 3 optimize the usage of xarray as is without flattening
##### Stage 3 optimize the usage of xarray as is without flattening
### This plots each timestamps missing pixels in a subfolder for easy visual rep
- This plots each timestamps missing pixels in a subfolder for easy visual rep
### Also logs in line with hydra for better maintanance and indicates the overall 14.98% missing pixels with a nside of 256 (Although nside 512 will go over above 721*1440 ERA5 dimensions its recommended to have slighltly lesser than overall pixels to disregard overlapped pixels in irregular guassian grid but can easily be experimented with in the config.yaml)
- Also logs in line with hydra for better maintanance and indicates the overall 14.98% missing pixels with a nside of 256 (Although nside 512 will go over above 721*1440 ERA5 dimensions its recommended to have slighltly lesser than overall pixels to disregard overlapped pixels in irregular guassian grid but can easily be experimented with in the config.yaml)
