AWS Tasks

---

This module contains a collection of tasks for interacting with AWS resources.

Note that all tasks require a Prefect Secret called "AWS_CREDENTIALS" that should be a JSON document with two keys: "ACCESS_KEY" and "SECRET_ACCESS_KEY".

S3Download

class

prefect.tasks.aws.s3.S3Download

(aws_credentials_secret=None, bucket=None, **kwargs)[source]

Task for downloading data from an S3 bucket and returning it as a string. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set a Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided, boto3 will fall back on standard AWS rules for authentication.
• bucket (str, optional): the name of the S3 Bucket to download from
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.s3.S3Download.run (key, aws_credentials_secret=None, bucket=None)[source]
Task run method. Args: key (str): the name of the Key within this bucket to retrieve aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided, boto3 will fall back on standard AWS rules for authentication. bucket (str, optional): the name of the S3 Bucket to download from Returns: str: the contents of this Key / Bucket, as a string

---

S3Upload

class

prefect.tasks.aws.s3.S3Upload

(aws_credentials_secret=None, bucket=None, **kwargs)[source]

Task for uploading string data (e.g., a JSON string) to an S3 bucket. Note that all initialization arguments can optionally be provided or overwritten at runtime.

For authentication, there are two options: you can set a Prefect Secret containing your AWS access keys which will be passed directly to the boto3 client, or you can configure your flow's runtime environment for boto3.

Args:

• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided, boto3 will fall back on standard AWS rules for authentication.
• bucket (str, optional): the name of the S3 Bucket to upload to
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.s3.S3Upload.run (data, key=None, aws_credentials_secret="AWS_CREDENTIALS", bucket=None)[source]
Task run method. Args: data (str): the data payload to upload key (str, optional): the Key to upload the data under; if not provided, a random uuid will be created aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY which will be passed directly to boto3. If not provided, boto3 will fall back on standard AWS rules for authentication. bucket (str, optional): the name of the S3 Bucket to upload to Returns: str: the name of the Key the data payload was uploaded to

---

LambdaCreate

class

prefect.tasks.aws.lambda_function.LambdaCreate

(function_name, runtime, role, handler, zip_file=None, bucket="", bucket_key="", object_version=None, description="", function_timeout=3, memorysize=128, publish=True, subnet_ids=None, security_group_ids=None, dead_letter_config=None, environment_variables=None, kms_key_arn="", function_tags=None, tracing_config="PassThrough", layers=None, aws_credentials_secret="AWS_CREDENTIALS", **kwargs)[source]

Task for creating a Lambda function.

Args:

• function_name (str): name of the Lambda function to create
• runtime (str): the identifier of the function's runtime
• role (str): the Amazon Resource Name of the function's execution role
• handler (str): the name of the method within your code that Lambda calls to execute your function
• zip_file (str): path to zip file containing code for Lambda function, either zip_file or (bucket and bucket_key) must be passed
• bucket (str): an S3 bucket in the same AWS region as your function
• bucket_key (str): the Amazon S3 key of the deployment package
• object_version (str, optional): for versioned S3 objects, the version of the deployment package to use
• description (str, optional): description of Lambda function
• function_timeout (int, optional): Lambda function timeout in seconds, default is 3 seconds
• memorysize (int, optional): amount of memory that Lambda function has access to in MB, must be a multiple of 64 MB, default is 128
• publish (bool, optional): set to True to publish the first version of the function during creation, defaults to True
• subnet_ids (List[str], optional): list of subnet ids for vpc configuration
• security_group_ids (List[str], optional): list of security group ideas for vpc configuration
• dead_letter_config (dict, optional): a dead letter queue configuration that specifies the queue or topic where Lambda sends asynchronous events when they fail processing
• environment_variables (dict, optional): key-value pairs of environment variables to pass to the Lambda function
• kms_key_arn (str, optional): the ARN of the AWS key management service used to encrypt your function's environment variables, if not provided, AWS Lambda uses a default service key
• function_tags (dict, optional): a list of tags to apply to the function, string to string map
• tracing_config (str, optional): set to Active to samle and trace a subset of incoming requests with Amazon X-Ray
• layers (List[str], optional): a list of function layers to add to the function's execution environment, specify each layer by its ARN
• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.lambda_function.LambdaCreate.run (aws_credentials_secret="AWS_CREDENTIALS")[source]
Task run method. Creates Lambda function. Args: aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY Returns: json: response from AWS CreateFunction endpoint

---

LambdaDelete

class

prefect.tasks.aws.lambda_function.LambdaDelete

(function_name, qualifier="", aws_credentials_secret="AWS_CREDENTIALS", **kwargs)[source]

Task for deleting a Lambda function.

Args:

• function_name (str): name of the Lambda function to delete
• qualifier (str, optional): specify a version to delete, if not provided, the function will be deleted entirely
• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.lambda_function.LambdaDelete.run (aws_credentials_secret="AWS_CREDENTIALS")[source]
Task run method. Deletes Lambda function. Args: aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY Returns: dict: response from AWS DeleteFunction endpoint

---

LambdaInvoke

class

prefect.tasks.aws.lambda_function.LambdaInvoke

(function_name, invocation_type="RequestResponse", log_type="None", client_context=None, payload="null", qualifier="$LATEST", aws_credentials_secret="AWS_CREDENTIALS", **kwargs)[source]

Task to invoke a Lambda function.

Args:

• function_name (str): the name of the Lambda funciton to invoke
• invocation_type (str, optional): the invocation type of Lambda function, default is RequestResponse other options include Event and DryRun
• log_type (str, optional): set to 'Tail' to include the execution log in the response
• client_context (dict, optional): data to pass to the function in the context object, dict object will be transformed into base64 encoded json automatically
• payload (bytes or seekable file-like object): the JSON provided to Lambda function as input
• qualifier (str, optional): specify a version or alias to invoke a published version of the function, defaults to $LATEST
• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.lambda_function.LambdaInvoke.run (function_name=None, payload=None, aws_credentials_secret="AWS_CREDENTIALS")[source]
Task run method. Invokes Lambda function. Args: function_name (str): the name of the Lambda funciton to invoke payload (bytes or seekable file-like object): the JSON provided to Lambda function as input aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY Returns: dict: response from AWS Invoke endpoint

---

LambdaList

class

prefect.tasks.aws.lambda_function.LambdaList

(master_region="ALL", function_version="ALL", marker=None, max_items=50, aws_credentials_secret="AWS_CREDENTIALS", **kwargs)[source]

Task to list Lambda functions.

Args:

• master_region (str, optional): for Lambda@Edge functions, the AWS region of the master function
• function_version (str, optional): the version of a function, default is 'ALL'
• marker (str, optional): specify the pagination token that's returned by a previous request to retreive the next page of results
• max_items (int, optional): specify a value between 1 and 50 to limit the number of functions in the response
• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.lambda_function.LambdaList.run (aws_credentials_secret="AWS_CREDENTIALS")[source]
Task fun method. Lists all Lambda functions. Args: aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY Returns: dict: a list of Lambda functions from AWS ListFunctions endpoint

---

StepActivate

class

prefect.tasks.aws.step_function.StepActivate

(state_machine_arn, execution_name, execution_input="{}", aws_credentials_secret="AWS_CREDENTIALS", **kwargs)[source]

Task for activating an AWS Step Function.

Args:

• state_machine_arn (str): the Amazon Resource Name (ARN) of the state machine to execute
• execution_name (str): the name of the execution, this name must be unique for your AWS account, region, and state machine for 90 days
• execution_input (str, optional): string that contains the JSON input data for the execution
• aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY
• **kwargs (dict, optional): additional keyword arguments to pass to the Task constructor

methods:
prefect.tasks.aws.step_function.StepActivate.run (aws_credentials_secret="AWS_CREDENTIALS")[source]
Task run method. Activates AWS Step function. Args: aws_credentials_secret (str, optional): the name of the Prefect Secret that stores your AWS credentials; this Secret must be a JSON string with two keys: ACCESS_KEY and SECRET_ACCESS_KEY Returns: dict: response from AWS StartExecution endpoint

---

This documentation was auto-generated from commit n/a
on May 14, 2020 at 21:12 UTC