datacatalog connections + datasets interplay is evaluated
As discussed with @boettcher1 , it could be a nice solution to make the pipelines more generic. The idea is to
- link datasets in data cat with connections in data cat.
- The stage-in pipeline would then resolve the dataset id,
- find a link to connection,
- instantiate this connection (datacat can be used as secrets backend
- use this connection to retrieve/move the data to target location