Allow check_fn of S3KeySensor to be used in more use cases

After #22737 the check_fn is part of S3KeySensor and can be used as additional check layer:

Lines 139 to 140 in 2bc1036

    
           if self.check_fn is not None: 
        
               return self.check_fn(files)

However if you will check the code you will see that it's tightly cuppled with the file size:

Line 137 in 2bc1036

files = [{"Size": obj["ContentLength"]}]

Line 127 in 2bc1036

files = [{"Size": f["Size"]} for f in key_matches]

However this doesn't have to be only for Size/ContentLength. We should support any key possible by the return object of head_object:

The task:
Make the Size/ContentLength parameterized (keep the default as is) but let the use decide how to construct the files list.

I'll take this one.

Navigation Menu