Link to chapter - https://serverless-stack.com/chapters/cognito-as-a-serverless-service.html
Cognito deployed with ServerlessFramework template is dependent on an Export from an S3 cloudformation Output that cannot be found…
I have defined a CICD pipeline on CodPipeline using CodeBuild to build my services declared with the ServerlessFramework CloudFormation templates using a BuildSpec.yml file.
I have an S3 service that exports the name of the bucket in the following manner:
service: qat-crm-uploads
custom:
stage: ${opt:stage, self:provider.stage}
provider:
name: aws
runtime: nodejs10.x
stage: dev
region: us-west-2
resources:
Resources:
... resource declarations are here
# Print out the name & Arn of the S3 bucket that is created
Outputs:
AttachmentsBucketArn:
Value:
Fn::GetAtt:
- AttachmentsBucket
- Arn
Export:
Name: ${self:custom.stage}-AttachmentsBucketArn
AttachmentsBucketName:
Value:
Ref: AttachmentsBucket
Export:
Name: ${self:custom.stage}-AttachmentsBucket
My AWS Cognito service declares a UserPool and an Identity Pool and the services is dependent on the Outputs exported by the S3 service above.
I have created a CodePipeline Stage that deploys the S3 service and exports the named Outputs as shown above and when the stage completes it then commences to build the Cognito User and Identity Pools.
Here is the flow of what the Pipeline stages look like on AWS:
S3Uploads --> Services that are working --> Cognito Service (Not able to read S3 Exports)
Here is how I am importing the named s3 exports into my Cognito serverless.yml template as a Cross Stack Reference:
Policies:
- PolicyName: 'CognitoAuthorizedPolicy'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action:
- 'mobileanalytics:PutEvents'
- 'cognito-sync:*'
- 'cognito-identity:*'
Resource: '*'
# Allow users to invoke the QAT CRM API
- Effect: 'Allow'
Action:
- 'execute-api:Invoke'
Resource:
Fn::Join:
- ''
-
- 'arn:aws:execute-api:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':'
- 'Fn::ImportValue': ${self:custom.stage}-ApiGatewayRestApiId
- '/*'
# Allow users to upload attachments to their
# folder inside of the s3 bucket created
- Effect: 'Allow'
Action:
- 's3:*'
Resource:
- Fn::Join:
- ''
-
- 'Fn::ImportValue': ${self:custom.stage}-AttachmentsBucketArn
- '/private/'
- '$'
- '{cognito-identity.amazonaws.com:sub}/*'
My CICD on CodePipeline is correctly importing the ${self:custom.stage}-ApiGatewayRestApiId value as described above and as it pertains to the Cross Stack Reference exported by the service invoking my Lamdas which I have defined separately also, but it is not importing my value for the ${self:custom.stage}-AttachmentsBucketArn which I am trying to import above to create the User and Identity Pools on Cognito.
Here is the error I am shown on CloudFormation that causes it to rollback and fail:
No export named dev-AttachmentsBucketArn found
How do I properly import the named export value from my S3 Upload service so that I can import it correctly in my Cognito Service to deploy on CodePipeline?
Any ideas what could be going wrong here? Let me know if I should add more of the code implementation here.
Thank you!
In a couple of days we’ll have a post on something similar, hopefully that’ll give you an idea.
Im getting the main service to export the API Rest Id’s and Arn’s so that the next dependent service can use it. And the Uploads bucket is in fact exporting the output correctly and I see the Bucket Arn in the terminal… The cognito service is getting the API Rest Id that is being out put by the main service deployed and yet it doesn’t pick up the S3 bucket Arn that in the Outputs for that service…
Any clue why cognito can’t access the bucket arn but it can still access the API GW Rest API Id?
Ill look out for the doc you have coming out and do you have any potential clues in the mean time that I can use to better trace this problem by any chance???
Thanks!!
Here is the link to the same question on stack overflow in case anyone wants to put the eventual resolution on there and link it back to this resource when you have it published to help improve your SEO and what not down the road.
I found the Error!!
Maybe this will help you with the next section you will publish. The Cross Stack reference to import the s3 bucket Arn should look like this instead:
# Allow users to upload attachments to their
# folder inside of the s3 bucket created
- Effect: 'Allow'
Action:
- 's3:*'
Resource:
Fn::Join:
- ''
-
- 'Fn::ImportValue': ${self:custom.stage}-AttachmentsBucketArn
- '/private/'
- '$'
- '{cognito-identity.amazonaws.com:sub}/*'
There was an added - in front of the Fn::Join statement that does not need to be there!
Thanks again guys!
Glad you figured it out. And thanks for reporting back in detail! Btw, this was the post I was referring to before.
Can you please discuss the discrepancy between the identity pool here and the one created above?
Specifically regarding the lack of the following code block above:
# Allow users to invoke our API
- Effect: 'Allow'
Action:
- 'execute-api:Invoke'
Resource:
Fn::Join:
- ''
-
- 'arn:aws:execute-api:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':'
- Ref: ApiGatewayRestApi
- '/*'
Comments like this
You’ll notice that we don’t have the IAM role here that allows access to our APIs. We are going to be doing that along side our API services.
Are pretty horrible for readability as a reference IMO. Where are you doing that alongside API services?
On the next page I do see Identity Roles referenced but it’s limited to Xray discussion. And in the repo I don’t see anything similar to the missing code block at all. Am I to assume we don’t need to grant those permissions anymore?
Thanks for the resource though it has been very helpful.
You are right, something is missing there.
That block that you pointed out is included in the service that is creating the API service here:
Where the policy itself is referenced from here:
I’ll add a note to the next chapter with this.
1 Like
Heads up to anyone reading through this - If you want to use multiple lambda services make sure to customize the policy name for each service.
For example
PolicyName: !Join [ "-", ['CognitoAuthorizedApiPolicy', '${self:custom.stage}', !Ref ApiGatewayRestApi] ]
Or create a dedicated service that handles your API policy and expose each ApiGatewayRestApi through a custom var and pull them into the policy.
Are you trying to create a separate policy for each service? Is there a reason why using the same policy across services doesn’t work?
Well the way its defined here it would be using each service’s specific ApiGatewayRestApi. So trying to use this specific example in a shared policy doesn’t work. It just overwrites itself with the latest created ARN.
A shared policy would work I think by exporting the ARN of each service and then importing them into the shared policy. But that would require a separate service because each of the lambda services would already have to be deployed in order for the imports to work.
This guide is amazing, and I am loving it so far.
However, I’m a bit concerned regarding frontend integration.
The frontend amplify configuration requires a lot of manual steps of copying the IDs and URLs of existing resources (like this cognito pools). I am wondering if there is a way to “inject” those values directly into the frontend, because otherwise you will have to do a lot of error-prone copy-pasting work to deploy the frontend to different environments.
Also, I tried to integrate this configuration as just a simple resource of my main sls application and I’m getting an error telling me CognitoAuthorizedApiPolicy - The specified value for roleName is invalid.
I just concat the resources section of the cognito service into my own serverless file and added the AWS iam roles also. I have no idea what the problem is, but if you have the will to help me I could post here the configurations that I tried. If you want me to open a separate thread for that, just let me know.
Thanks!
Well, found the solution myself for the latter problem.
I just updated the section CognitoAuthorizedApiPolicy: to reference the pools like this:
CognitoAuthorizedApiPolicy:
...
Roles:
- Ref: CognitoAuthRole
Previously I was using Fn::GetAtt:
So my only concern right now is the manual steps on the frontend.
Thanks for this great guide!
In the guide, we later use the same API Gateway project for multiple services and only use a single API Gateway project across the entire app. Does that make sense?
Glad you figured it out.
For the frontend config idea, are there any other solutions in the non-Serverless world that tackle this?
Do you mean that you just use the same urls for several services? LIke, same api for staging, dev, and prod?
But everytime that you recreate them they will change right?
In any case, I just wrote an script that parses sls info -v output and reads the different arns from there. Then I put them on environment variables and run the build from there.
Well the difference services will have the same base URL. You can see this when we deploy multiple services here — https://serverless-stack.com/chapters/deploy-the-api-services-repo.html
But the various stages (dev, prod, staging, etc.) will have different URLs.
How would the yml file look if we wanted to the user to upload their attachments to a public folder?
Hmm there’s a bit more than just this YAML. I think a few folks in these forums have talked about it. But we’ll be looking at adding a chapter on this.