Skip to content

Switch to typescript and response streaming#162

Merged
joehoyle merged 37 commits into
masterfrom
v3-branch
Nov 20, 2023
Merged

Switch to typescript and response streaming#162
joehoyle merged 37 commits into
masterfrom
v3-branch

Conversation

@joehoyle
Copy link
Copy Markdown
Member

@joehoyle joehoyle commented Jul 27, 2023

This is a major version, with the following:

  • Switch the repo to use Typescript, which provides a lot more safety around the Lambda request args info, args etc.
  • Upgrade to Node v18
  • Use Lambda Response Streaming to we can return larger files and improve TTFB
  • Use newer API Gateway/Lambda Function URL request type to support Lambda function URLs
  • Remove support for CloudFormation
  • use the AWS SAM stack for local dev and testing
  • Added support for GIF resizing
  • New Readme / testing docs
  • Auto build the lambda file on publish release
  • Switch to Github Actions for testing

Todo:

  • Support for proxying signed requests
  • local server solution for local-server

@joehoyle joehoyle requested a review from rmccue July 28, 2023 14:01
@joehoyle joehoyle marked this pull request as ready for review July 28, 2023 14:02
Comment thread package.json Outdated
"test": "docker run --rm -v `pwd`:/var/task -it --entrypoint='node' public.ecr.aws/lambda/nodejs:14 /var/task/test-filesize/index.js",
"update-test-fixtures": "docker run --rm -v `pwd`:/var/task -it --entrypoint='node' public.ecr.aws/lambda/nodejs:14 /var/task/test-filesize/index.js --update-fixtures",
"build-zip": "rm lambda.zip; zip -r --exclude='node_modules/aws-sdk/*' --exclude='node_modules/animated-gif-detector/test/*' lambda.zip ./node_modules/ index.js proxy-file.js lambda-handler.js",
"build": "sam build -u",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sam actually added as a dependency? (Is it in aws-lambda?)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's part of https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html. I think we need to document it in CONTRIBUTING.md though

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK documented

Comment thread src/lambda-handler.ts Outdated
Comment thread src/lambda-handler.ts Outdated
Comment thread src/lambda-handler.ts Outdated
Comment thread src/lib.ts Outdated
Comment thread src/lib.ts Outdated
}

// add invalid args
resolve({ data, info: { ...info, errors: errors.join(';') } });
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return errors: string[]?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's just what it did already and I didn't want to change the interface

Comment thread src/server.ts Outdated
Comment thread src/server.ts
Comment thread src/server.ts Outdated
Comment thread template.yaml
Handler: dist/lambda-handler.handler
Environment:
Variables:
S3_BUCKET: hmn-uploads
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this use a dummy name if it's just a template? Don't want to risk it accidentally being used

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is meant to be used to be able to do the integration test. There's no permissions to read / write the bucket it's just public

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Can we use the test bucket in that case? (Per compliance, not using prod data for testing/development.)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is no different than having a test which does curl http://humanmade.com/robots.txt to check it's working, so I don't think that really falls under using prod data for testing / development. That's more to do with copying prod data to development, so it leaks data potentially to people who only have access to "test data".

@rmccue
Copy link
Copy Markdown
Member

rmccue commented Aug 8, 2023

Also, the coding style here is all over the place. Tachyon was already a bit not great on this, but would be a good time to clean that up

joehoyle and others added 7 commits August 9, 2023 14:34
Co-authored-by: Ryan McCue <me@ryanmccue.info>
Co-authored-by: Ryan McCue <me@ryanmccue.info>
Co-authored-by: Ryan McCue <me@ryanmccue.info>
Co-authored-by: Ryan McCue <me@ryanmccue.info>
@joehoyle
Copy link
Copy Markdown
Member Author

joehoyle commented Aug 9, 2023

Ok just need to decide on coding standards. At the moment the whole project uses Prettier with tabs. Do we want to use hm eslint-config? It hasn't been updated in 3 years so I wasn't sure whether we actually wanted to use that or not.

@joehoyle
Copy link
Copy Markdown
Member Author

joehoyle commented Aug 9, 2023

@rmccue ok ready for another review.

@joehoyle
Copy link
Copy Markdown
Member Author

joehoyle commented Sep 4, 2023

Fixes #3

Comment thread src/lib.ts
'X-Amz-Date',
'X-Amz-Security-Token',
] as const;
const presignedParams: { [K in ( typeof presignedParamNames )[number]]?: string } = {}; // eslint-disable-line no-unused-vars
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would probably be a bit cleaner if you split the type out I think, since it's relatively complex?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hoisting it out of the function I don't think really does much to reduce the complexity

Comment thread src/lib.ts Outdated
Comment thread src/lib.ts Outdated
Comment thread src/lib.ts
Comment thread src/lib.ts
@joehoyle joehoyle requested a review from rmccue September 21, 2023 07:22
@joehoyle joehoyle merged commit 8702dfa into master Nov 20, 2023
@joehoyle joehoyle deleted the v3-branch branch November 20, 2023 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants