Ytukyg

I am getting the following error on one version of boto3, and not on another, without changing code:

An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I encounter this error on boto3 1.7.74 (with botocore 1.10.84), in the aws lambda environment. I can reproduce the error if I install the same boto3 version with pip in my dev environment.

The problematic code is quite simple:

# python3



s3 = boto3.client('s3')

base64_md5 = hex_to_b64(md5_hexdigest)

extra_args = {

    'ContentType': 'application/octet-stream',

    'ContentMD5': base64_md5, # "dsE2vA1oCr2XjZk0BwwB5Q=="

    'ContentLength': content_length, # 62914560

    'Metadata': {}

}

try:

    completed = s3.put_object(Bucket=bucketname, Key=keyname, Body=inputfp, **extra_args)

Sometimes these signature issues are associated with passing bad credentials to the code. My credentials are stored under ~/.aws/, and I have AWS_PROFILE exported in my dev environment. Otherwise, I'm not passing any keys or tokens to my code.

In the code above, inputfp is a non-seekable() file object (I'm feeding boto3 the raw response body of a GET request to a local webserver). Content-Length and Content-MD5 are known by the time I feed it to boto3. My particular test file happens to be 60MB (62914560bytes) of zeroes (dd if=/dev/null bs=1M count=0 seek=60 of=zeroes). The md5 is 76c136bc0d680abd978d9934070c01e5, which corresponds to the b64-encoded value dsE2vA1oCr2XjZk0BwwB5Q==, passed via the ContentMD5 parameter.

I do not encounter this error on boto3 v1.9.35 (with botocore v1.12.35) via pip on my dev box. Similarly, I do not get the error if I include that version in my lambda code zip.

I suspect boto 1.7.74 / botocore 1.10.84 doesn't handle stream fds as well as a more recent version, but it's not obvious to verify -- the botocore library appears to be almost entirely procedurally generated from stubs. Anyone knows where in botocore the signature headers are produced by any chance? -- I would compare the two versions.

I will add a few more wrappers on my file object, and see where possibly it might go wrong.