Menu

1 / 6

S3 Files Killed My Least Favorite Lambda Pattern

DEV Community: lambda·Mwanza Simi·5 months ago
#18eGdd5b
#dev#files#file#function#bucket
Reading 0:00
15s threshold

Every Lambda function I have written that touches S3 has the same three lines of plumbing:

s3 . download_file ( bucket , key , " /tmp/input.csv " ) process ( " /tmp/input.csv " , " /tmp/output.csv " ) s3 . upload_file ( " /tmp/output.csv " , bucket , output_key )

Enter fullscreen mode

Exit fullscreen mode

Download. Process. Upload. Clean up /tmp . Handle the edge case where /tmp is full from a previous invocation. Handle the edge case where the download fails halfway. Handle the edge case where you run out of the 10 GB ephemeral limit because someone uploaded a file larger than you expected.

S3 Files makes all of that go away. You mount the bucket at /mnt/workspace and use open() . The file is right there. You write the output.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.