PhysicalResourceId in AWS CloudFormation

What are PhysicalResourceId and LogicalResourceId?

Many of us use Cloudformation for deploying our infrastructure in AWS. We love it for it’s simplicity and transparency. Also Cloudformation is extensible and one of the ways how it can be extended is Custom Resources. Let’s talk about one of the intricacies of using Custom Cloudformation resources in connection to the PhysicalResourceId.

When you deploy a Cloudformation stack, every resource you describe in the template gets a LogicalResourceId, which identifies the Cloudformation section that describes the resource, in the below case it will be ActiveDirectoryController1:

Also it gets a PhysicalResourceId, which identifies the deployed resource itself. In the above case the PhysicalResourceId will be the EC2 instance id:

Why is PhysicalResourceId important to us?

When Cloudformation updates properties of a resource, that can’t be changed (for example if you are updating the subnet on an EC2 instance), it replaces the resource and the new resource will have a different PhysicalResourceId. When a resource is being replaced during a deployment Cloudformation first creates a new resource and then, during the clean-up phase, it deletes the old one.

Now Custom Cloudformation Resources

When you want to have Cloudformation deploy something unusual, you go for Custom Resources. Custom resource is handled by a Lambda function (or an SNS Topic, but it all comes down to the same thing), that accepts a request from Cloudformation, performs the provisioning actions and then sends the result back to Cloudformation by invoking a pre-signed S3 URL that is found in the request. Normally you would use a library like cfn-response-promise to not bother with HTTP and pre-signed URLS:

await cfnResponse.send(event, context, cfnResponse.SUCCESS, {Property: 'Value'}, physicalResourceId); 
return null; 

When you update a custom resource Cloudformation will recognise that the resource is being updated in-place when your Lambda function returns the same PhysicalResourceId. On the opposite if you return a new PhysicalResourceId, that is different from the one the resource had before the update, Cloudformation will know that the resource is being replaced and it will try to delete the “old resource” during the clean-up phase

What happens if you don’t care too much about the PhysicalResourceId in your reply and don’t include it in the cfnResponse.send call? Well, all the off-the-shelf Cloudformation response libraries I have seen, including the reference AWS cfn-response implementation, send the Cloudwatch Log stream id as the PhysicalResourceId. Here is how cfn-response-promise handles it (why do they still use var declarations btw?):

var responseBody = JSON.stringify({ 
    Status: responseStatus,
    Reason: "See the details in CloudWatch Log Stream: " + context.logStreamName,     
    PhysicalResourceId: physicalResourceId || context.logStreamName,     
    StackId: event.StackId,     
    RequestId: event.RequestId,     
    LogicalResourceId: event.LogicalResourceId,     
    Data: responseData 
}); 

This will inevitably lead you to the situation when after a re-deployment of the Lambda function it will create a new Cloudwatch Log stream and will start returning a new PhysicalResourceId regardless of whether you are replacing or updating the resource. And then, as we discussed above, Cloudformation will try to delete the “old resource”, which is probably the only resource you have, as it will think that you are replacing the resource.

What does that all mean to me?

When you have a custom resource that simply returns you some information (like we do in the cloudformation-vpcinfo-lambda, for example) it doesn’t make too much difference as you are not operating with any real resources and do not care if Cloudformation decides to “delete” your pseudo-resource. In this scenario you probably can rely on the defaut behaviour of cfn-response and use Cloudwatch Log stream id or whatever else you find aesthetic.

The story becomes different though if your custom resource does create something real which you need to keep track of. In this case you should think through handling of the PhysicalResourceId. Normally it’s not such a complex task because whatever physical thing you create will have a natural way of identifying itself. As an example I can bring up a mechanism we used to deploy StackSet stack instances (oh what a clunky terminology!) into AWS accounts by means of custom resources, i.e. we had a Lambda function we invoked from a Cloudformation stack that would trigger a stackset deployment into a specified AWS account. The physical thing it created was the stack instance and an expression that would uniquely identify a stack instance and therefore could be used as the PhysicalResourceId could be this: stack-set-name:AWS account ID:AWS region, which, when you fill it out, would look like: sharedservice-vpc-set:123456789012:ap-southeast-2.

Using a consistent scheme for PhysicalResourceIds will let you avoid a very unpleasant surprise when you decide to update a Cloudformation stack and find your custom resources blown away by Cloudformation as it once happened to me.

Good luck, Alex.