Automating the Automators: Dynamic AWS Lambda Creation in Event-Driven Systems

Error Handling + Logging

Right now, the demo assumes everything goes smoothly, but that’s rarely the case in the real world. When creating infrastructure on the fly, especially with Lambda, it’s vital to build for the moments when things don’t work.

Handle Errors Gracefully

When calling create_function, wrap it in a try/except block to catch issues like missing permissions, invalid parameters, or naming conflicts. These errors happen, and failing gracefully can mean the difference between a smooth recovery and a broken workflow.

Here’s a simple example:

import logging
from botocore.exceptions import ClientError

logger = logging.getLogger()
logger.setLevel(logging.INFO)

try:
    response = lambda_client.create_function(**params)
    logger.info(f"Created function: {params['FunctionName']}")
except ClientError as e:
    logger.error(f"Failed to create function {params['FunctionName']}: {e}")

Log What Matters

Instead of print(), use the logging module built for this. These logs become your breadcrumbs when debugging issues in CloudWatch. At minimum, log:

• The event input (trimmed if it’s too big)
• The function name created
• Any exception messages or stack traces

Plan for Failures

If the function creation fails, don’t just drop the error. Route the failed event to an EventBridge dead-letter queue (DLQ) or an SQS queue for later retries. You can also configure an EventBridge rule with exponential backoff to automate the retry logic. This adds resilience and allows your automation to recover gracefully, without requiring manual intervention.

In short: Don’t just hope it works. Expect failure, log intelligently, and build with recovery in mind.

Function Code Explained

For this example, we will use Python 3.11.0. The same approach will work with whatever language you choose to use with Lambda. The same IAM permission set above in prerequisites is required regardless of the application code language. For this code to run, you will need the following packages imported into your source function. 

• boto3 – The official AWS SDK for Python is pre-installed in all Python Lambda runtimes. This lets you interact with AWS services like S3, DynamoDB, and Lambda.
• json – This is a standard Python library, so it’s automatically included in all Python runtimes.
• zipfile – This is part of Python’s standard library and is available by default in Lambda’s Python runtimes.

Setup Lambda Variables

• region = <your-desired-aws-region>

• lambdas_iam_role = "arn:aws:iam::<aws-accountid>:role/service-role/<your-target-lambda-execution-role-name>"

  • This is an IAM role that you’d like to attach to the target Lambda that gets created. It can be the same or different from the source execution role, but will likely differ based on your needs. 

• myvar = "myvariable"

  • This can be any variable you need to pass to the target lambda function that will be created in response to the event. You can add more as needed. Name them whatever you like. 

• template_name = "my-target-template"

  • This should be the name you want your target lambda created with. This is the name that will show in the Lambda console.

Setup boto3 Clients

You’ll need two clients from the boto3 SDK for this to execute correctly:

• lambda_client = boto3.client('lambda', region_name=region)

  • Notice it initiates the boto3 client for lambda using the region variable set above

• s3_client = boto3.client('s3', region_name=region)

  • Notice it initiates the boto3 client for S3 using the region variable set above

The Lambda Function

This is the function that will be called from the lambda handler to do the work we want done. Specifically:

1. Define the target lambda code, including the lambda handler. 
2. Zip the Lambda function and save locally to the source Lambda environment in /tmp directory.
3. Call the lambda client we created and create the lambda function with the zipped code.
4. Attach the target execution role when the target Lambda is created
5. Define the necessary lambda configuration for the target function

Let’s review each section:

def create_lambda_function():

lambda_code = textwrap.dedent(f"""\
import json
import boto3

region = '{region}'
template_name = '{target_name}'
lambda_client = boto3.client('lambda', region_name=region)

def lambda_handler(event, context):
    print(json.dumps(event))
    parsed_event = json.loads(json.dumps(event))
    user_id = parsed_event.get('user_id', 'unknown')
    print(f"Processed user ID: {{user_id}}")
    return {{
        "status": "ok",
        "user_id": user_id
    }}
""")

with zipfile.ZipFile('/tmp/lambda_code.zip', 'w') as zfs:
       zip_info = zipfile.ZipInfo('lambda_function.py')
       # Write the Lambda code to the ZIP file with the specified permissions
       zfs.writestr(zip_info, lambda_code)

lambda_client.create_function(
       FunctionName=template_name,
       Runtime='python3.11',
       Role=lambdas_iam_role,
       Handler='lambda_function.lambda_handler',
       Code={
           'ZipFile': open(b'/tmp/lambda_code.zip', 'rb').read()
       },
       Environment={
           'Variables': {
               'mylambdavariable': myvar,
           }
       },
       Architectures=[
           'arm64',
       ],
       Description=f'Lambda function',
       Timeout=30,
       MemorySize=256
   )

def lambda_handler(event, context):
   parsed_event = json.loads(json.dumps(event))
   print(parsed_event)
   create_lambda_function()

Deploy the Target Function to the Lambda Service

For simplicity, we will deploy the function from the CLI. You can log in to the Lambda console to see the resulting deployment. Prerequisites for this stage are having AWS credentials configured locally for the account and the default region to which you’d like to deploy the Lambda. For details on configuring AWS credentials locally (if you haven’t done so), visit this link.

1. Download the file function here
2. Create the role based on the policy above.
3. Be sure to replace , , and any placeholders in the zip path with actual values from your environment.
4. Run this command at your CLI. This will create a file called function.zip in the directory you run it in from the CLI. Run it in the same directory as your lambda_from_lambda.py for simplicity.

   zip -j ./function.zip /<path to lambda .py>/lambda_from_lambda.py

5. Then you can run this CLI command to deploy. Make sure your reference to the zip file created above is correct.

   aws lambda create-function \
       --function-name create-lambdas-lambda \
       --runtime python3.11 \
       --role arn:aws:iam::<account-id>:role/service-role/<your role name> \
       --handler lambda_from_lambda.lambda_handler \
       --zip-file fileb://function.zip \
       --timeout 30 \
       --memory-size 256 \
       --architecture arm64
   

   This will deploy the Lambda function to the Lambda service in AWS. You can log in to the Lambda console, and you should see a new function there in the region you set as default in your aws credentials configuration. The lambda function will be called create-lambdas-lambda if you didn’t change it in the example code.

   

   Let’s Test the Secondary Lambda Creation

   This code is pretty simple and doesn’t have a lot of utility beyond creating the secondary lambda in response to an event. But you can begin to see how you can extend the code in either lambda to have more utility based on your use case, with the simple pattern to create a lambda with a lambda in response to an event as the foundation. To test this code, we’ll simply run a test in the Lambda console to see if the secondary Lambda gets created in response to our test event.

6. In the Lambda console, click on the name of the create_lambdas_lambda to open it.
7. Click the test from the menu at the top of the editor.
   You can accept all the defaults on this page, with the only change being to give the test event a name in the “Event name” field. In this example, the Lambda does not process any specific content from the event payload, and it simply triggers the creation of the secondary Lambda.
   However, this pattern is highly extensible. You can imagine passing additional fields through the event, such as configuration settings or user-specific data, which the primary Lambda could use to dynamically customize the behavior, logic, or environment variables of the generated function. This makes the approach well-suited for building personalized automation or per-tenant infrastructure at runtime.
   
8. Once you’ve named the test event, click the orange test button in the top right. This should fire the lambda test and create a green header above the editor showing success. A red header will display error information in the detail view that you can troubleshoot. If IAM permissions were configured correctly above, the lambda should fire.
   
9. Return to the main lambda console, and you should now see a second lambda function called “dynamic-function” created from the firing of the test event in the first function.
   
10. If you click the “dynamic-function-xxxx” function name, you should see the code in the ‘f-string’ of the primary lambda in the code editor. This secondary lambda code doesn’t have any utility since it merely prints the payload that was passed in from the test event, but you can see that creating a lambda from a lambda in response to an event worked!

Security Best Practices

Security should be foundational, especially when dealing with dynamically created functions. Here are a few key practices to follow:

• Use least privilege IAM roles: Assign the minimum permissions necessary to the parent and child Lambda functions. This reduces the blast radius if credentials are compromised.
• Avoid injecting unvalidated content into f-strings: If you’re building code or commands dynamically, never insert untrusted input directly. It opens the door to remote code execution vulnerabilities.
• Encrypt environment variables: If your functions rely on secrets or sensitive configuration, store them securely, using AWS KMS or a secrets manager to encrypt environment variables.

Threat Matrix: Risks of Dynamic Lambda Creation

While dynamic function creation enables agility and flexibility, it also introduces specific security and operational risks. Below is a simplified threat matrix to help assess potential vulnerabilities and plan mitigations:

Conclusion

Dynamically creating AWS Lambda functions isn’t a widely adopted pattern, but when applied in the right context, it unlocks unique capabilities. Whether you’re enabling per-tenant compute, dynamically transforming event data, or deploying ephemeral developer environments, this approach provides on-demand scalability, automation-driven flexibility, and reduced operational overhead.

Next Steps

To explore this pattern further:

• Adapt the sample code to your use case, whether tenant provisioning, dynamic event handling, or sandbox creation.
• Use AWS Step Functions to orchestrate more complex workflows involving multiple dynamic Lambdas.
• Apply input validation and secure coding practices to mitigate code injection or configuration risks.
• Log and monitor function creation and execution to maintain observability and governance.

Have questions, feedback, or a use case to share? Leave a comment, we’d love to hear how you’re using dynamic serverless automation in your architecture.

Authors: David Ernst, Principal Architect; Jeff Carson, CTO at ClearScale