SciHub S2 Download Function
This function is meant to be triggered using an CloudEvent (of type io.y22.eo.s2.scihub.download_product
).
The function expects to receive a CloudEvent containing as data the description of an Sentinel-2
product, as generated by the discover-s2-data
function.
Requirements
This function requires the following environment variables to be set:
SCIHUB_USERNAME
SCIHUB_PASSWORD
-
BACKOFF_MAX_TIME
: Backoff maxtime in case of dhus issues -
BACKOFF_MAX_TRIES
: Maximum number of retries in case of dhus issues -
DESTINATION_PATH
: Destination path inside the S3 bucket. All data will be stored beneth this directory -
KEEP_ORIGINAL_PRODUCT_FILE
: Store in the bucket the original file. Values ofyes
,True
or1
are interpreted asTrue
, everything else asFalse
-
UPLOAD_EXTRACTED_PRODUCT
: Should the extracted product be stored inside the bucket ?. Values ofyes
,True
or1
are interpreted asTrue
, everything else asFalse
- (optional)
S3_ENDPOINT_URL
: S3 Endpoint to be used. -
S3_DESTINATION_BUCKET
: The destination bucket name -
S3_KEY
: S3 authentication key -
S3_SECRET
: S3 secret
Return
This function return an object of type io.y22.eo.s2.scihub.product_downloaded
containing modified product metadata pointing to the location of downloaded data instead of the original data.
Deployment
kn
Using kn service create -n functions --force download-s2-data -p 8080 --image registry.terrasigna.com/internal/infrastructure/functions/download-s2-data:latest --scale-min 0 --scale-max 20 --pull-policy Always -e SCIHUB_USERNAME=${SCIHUB_USERNAME} -e SCIHUB_PASSWORD=${SCIHUB_PASSWORD} -e DESTINATION_PATH=${DESTINATION_PATH} -e S3_DESTINATION_BUCKET=${S3_DESTINATION_BUCKET} -e S3_ENDPOINT_URL=${S3_ENDPOINT_URL} -e S3_KEY=${S3_KEY} -e S3_SECRET=${S3_SECRET}
Using offline configuration
Equivalent service definition:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
creationTimestamp: null
name: download-s2-data
namespace: functions
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/max-scale: "20"
autoscaling.knative.dev/min-scale: "0"
client.knative.dev/updateTimestamp: "2022-05-16T20:48:58Z"
client.knative.dev/user-image: registry.terrasigna.com/internal/infrastructure/functions/download-s2-data:latest
creationTimestamp: null
spec:
containers:
- env:
- name: SCIHUB_USERNAME
value: ${SCIHUB_USERNAME} # To be replaced
- name: SCIHUB_PASSWORD
value: ${SCIHUB_PASSWORD} # To be replaced
- name: DESTINATION_PATH
value: ${DESTINATION_PATH} # To be replaced
- name: S3_DESTINATION_BUCKET
value: ${S3_DESTINATION_BUCKET} # To be replaced
- name: S3_ENDPOINT_URL
value: ${S3_ENDPOINT_URL} # To be replaced
- name: S3_KEY
value: ${S3_KEY} # To be replaced
- name: S3_SECRET
value: ${S3_SECRET} # To be replaced
image: registry.terrasigna.com/internal/infrastructure/functions/download-s2-data:latest
imagePullPolicy: Always
name: ""
ports:
- containerPort: 8080
resources: {}