Airflow Amazon provider S3Hook().download_file() fail when needs encryption arguments (SSECustomerKey etc..)  #25833

Apache Airflow version

2.3.3

What happened

Bug when trying to use the S3Hook to download a file from S3 with extra parameters for security like an SSECustomerKey.

The function download_file fetches the extra_args from self where we can specify the security parameters about encryption as a dict.
But download_file is calling get_key() which does not use these extra_args when calling the load() method here, this results in a botocore.exceptions.ClientError: An error occurred (400) when calling the HeadObject operation: Bad Request. error.

This could be fixed like this:
load as says boto3 documentation is calling S3.Client.head_object() which can handle **kwargs and can have all the arguments below:

response = client.head_object(
    Bucket='string',
    IfMatch='string',
    IfModifiedSince=datetime(2015, 1, 1),
    IfNoneMatch='string',
    IfUnmodifiedSince=datetime(2015, 1, 1),
    Key='string',
    Range='string',
    VersionId='string',
    SSECustomerAlgorithm='string',
    SSECustomerKey='string',
    RequestPayer='requester',
    PartNumber=123,
    ExpectedBucketOwner='string',
    ChecksumMode='ENABLED'
)

An easy fix would be to give the extra_args to get_key then to load(**self.extra_args)

What you think should happen instead

the extra_args should be used in get_key() and therefore obj.load()

How to reproduce

Try to use the S3Hook as below to download an encrypted file:

from airflow.providers.amazon.aws.hooks.s3 import S3Hook

extra_args={
            'SSECustomerAlgorithm': 'YOUR_ALGO',
            'SSECustomerKey': YOUR_SSE_C_KEY
        }

hook = S3Hook(aws_conn_id=YOUR_S3_CONNECTION, extra_args=extra_args)
hook.download_file(
            key=key, bucket_name=bucket_name, local_path=local_path
        )

Operating System

any

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct