Followup questions to the above trail. Please elaborate on: “With a Lambda Function you can download the contents of the export anywhere in AWS”
Once we finish setting up the Policy with the Lambda ARN where will the export of 1000 devices go into in AWS? AWS CONFIG in the AWS LAMBDA node in Losant allow access to AWS but it does not seem to set a location within AWS to place the export at. Please elaborate on the final location of the export.
I defined the data method under the AWS LAMBDA node LAMBDA CONFIG as workflow payload. Will all 1000 device workflow payloads be exported? with what extension name?
I want to take a step back a bit to make sure we are leading you down the right path. Earlier in this thread, we mentioned that this route was a more advanced option.
To accomplish this goal, we recommend learning and taking a look at AWS and learning a bit more about:
Here is a great video and resoruce online that helps pull some of these pieces together:
I recommend taking a deeper dive into AWS’s resources and going through some basic Lambda examples.
I’m recommending this route because of these questions:
The answers here depend on how you want to configure your Lambda function and application behavior.
However, after you digging to Lambda a bit, here is the answer to your questions:
The export doesn’t generate raw data. It generates a link to the raw data. You can make an HTTP request within the Lambda Function. And then, within the function, you’ll have access to the data, and you can decide where to forward it from there within your AWS environment.
I understand that the AWS Lambda function will need to be configured to reference a URL (the Webhook Trigger URL I assume?) and pull the actual data export file (generated by Losant API). The Daily exported file (triggered by Losant Timer Node) will be pulled by AWS from Losant with a Lambda Function. Only once the AWS Lambda Function is configured can the Policy ARN be completed.
My followup questions are the following:
How will this file look like? I think the format is set by Losant, please confirm and explain how a 1000 device file be formatted? How can I choose only the device ID and data to be exported for all data within a 24 hour period?
I will need to synchronize the Pull (by AWS Lambda Function) and the Devices:Export generated by LOSANT API and Timer nodes workflow. Is the below timeline guaranteed for 24 hours gathered data of 1000 device export file from 12:00(previous day)-12:00(current day)?
I will post the AWS Lambda function I eventually used from within AWS to pull the LOSANT API node exported file through the AWS LAMBDA node Webhook URL + Appropriate AWS CONFIG. I am looking for an AWS Lambda function that will pull it to an S3 Bucket now. However, if you have experience with similar AWS Lambda functions and wish to share it will be greatly appreciated.
With respect to the Losant aspect of connecting the AWS Lambda Node:
Please confirm the URL from which AWS Lambda Function pull the Exported File will be the webhook URL I generated in Webhook node in the workflow above.
From the documentation you refer to regarding my question about Devices:Export it seems the file that the AWS Lambda Function will pull through the URL will be with a CSV extension similar to the file generated by the import/export (Can you confirm that?)
This is not what I was expecting, as you can see this look like a general summary of the devices. I would like to export all devices data attributes collected for the last 24hours (from noon the day before to noon at the time of the Lambda pull) into the Device:Export. I tried to understand this through the Rest API Documentation but could not find how to fine-tune the Device:Export resource & action.
Do I need to place the Advanced Device Query Schema in the Losant API node? If yes, can you provide a version that query all device’s data attributes?
Now that the AWS Lambda function is able to pull the Devices:Export I received a payload with a “downloadUrl” that store the CSV file in a google cloud client library: https://storage.googleapis.com/
Can you confirm this storage in google cloud is temporary? What is the life span of my exported file at Google?