There is an option to download a file in the aws s3 node. However, it does not actually download the file or provide me with the data from the file in any way. How can I either a) download then upload the file or b) automatically access the internal contents of the file? Ultimately my goal is to automatically transfer much of this data into a google sheet to be shared elsewhere.
What is the error message (if any)?
Please share your workflow
My current workflow starts with a Schedule Trigger. Set to run the workflow once per week. Next is AWS S3 Bucket Search. Next is AWS S3 File Download. I choose the files using the file key given from the bucket search step. Next is Google Sheets. In theory this is where I append the data from the S3 file into the google sheet.
(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)
Share the output returned by the last node
The output simply puts the data from the search bucket node into the google sheet. This includes file key, etag, etc. Instead I need it to append the data from the s3 files.
This isn’t quite what Google Sheet needs as per my comment above, but the download should still work of course. Are you not getting this binary file? Any errors?
Thanks @MutedJam for the response.
Yes you are correct that Google sheets and AWS are both working as expected. My issue is that the Download option in aws does not help with automation. My goal is to automatically pull files from the aws bucket and write them to a google sheet. Clearly it is not possible with this straightforward solution. I am wondering if there is a way to automatically have n8n press the download button so that I can access it from my computer? Or any way to write to the sheet file with the data from the s3 bucket. Thanks.
Hi @eaferstl, Google Sheets would not require downloading images in order to add them to a sheet. Assuming your buckets have suitable permissions set you’d need to provide the public URL (something like https://$bucket.s3.$region.amazonaws.com/$key iirc) to Google Sheet as shown in the linked thread.
I am wondering if there is a way to automatically have n8n press the download button so that I can access it from my computer?
n8n is designed to run its workflows in the background. So downloading a file from the execution view is more of a debugging task rather than something happening regularly. For this reason I don’t think there will be any functionality to automatically send binary files to the user’s browser when manually executing workflow anytime soon.
You can of course still automate the download part of the process quite easily, it’d just not happen in the n8n UI itself. An example workflow implementing such logic would be available in our workflow template library:
Whenever someone visits the webhook URL, a file is being retrieved from a server and then downloaded in the user browser. You can easily use this template, and swap the HTTP download with your S3 download logic.
@MutedJam ok I am now experiencing an issue with the download option in aws s3. The node just keeps running with no output. Another person I am working was still able to download.
Endlessly running nodes typically suggest a networking problem between your n8n server and your browser. This often comes down to the reverse proxy configuration, though this wouldn’t be a problem on n8n cloud.
Does this issue persist when accessing your n8n instance using another browser or from another network?
Following up from my last message, I had a quick search for your cloud instance and it seems it’s frequently running out of memory.
So I suspect the binary data you’re processing in your workflow might just be too much, and the reason you don’t see any progress is that your instance crashes in the background
You might want to check out our documentation page on memory-related problems for more information, and for example consider processing less data in each execution.
