Cross-region/account references

[rix0rrr]

We were in the progress of defining how to transparently make this work.

Tracking it here because it's one of the most important things to pick up post-release.

[rix0rrr]

Use case: cross-region replication in combination with encryption.

• Stack A -- defines the target bucket + a KMS key
• Stack B -- defines the source bucket + an IAM role to use for replication.

For replication, the IAM role must must reference both buckets (local reference + one-way cross-stack reference)

To add encryption, the role must also reference the KMS Key, and the KMS key must also reference the role (bidirectional reference between stacks!)

[rix0rrr]

Note to self: if we build an ARN from a well-known name, we must not forget to manually add a dependency between the resources involved.

Probably, we should just link references between resources, which may then automatically translate into references between stacks.

[skinny85]

Use case: cross-account CodePipeline.

Below is an example of a hacky way of defining a cross-account CodePipeline using CDK code today.
This follows the official blog post on the subject, but is a little simpler:
in the blog post, there were a total of 4 accounts, 2 used for deployment (test and prod).
I'm modelling a 2-stage Pipeline only, so I have 2 accounts: one with the CodeCommit repository in it,
and another where the CodeBuild project and the Pipeline itself live.

import { AccountPrincipal, App, ArnPrincipal, PolicyStatement, Stack } from 'aws-cdk';
import * as cp from 'aws-cdk-codepipeline';
import * as cb from 'aws-cdk-codebuild';
import * as cc from 'aws-cdk-codecommit';
import * as s3 from 'aws-cdk-s3';
import * as iam from 'aws-cdk-iam';
import * as kms from 'aws-cdk-kms';
import { codecommit } from 'aws-cdk-resources';
import { HackyIdentity } from "./hacky-identity";

const app = new App(process.argv);

const pipelineAcc = '123456789012';

/*************** the CodeCommit Repository Stack *************************/

const repoAcc = '012345678901';
const repoAccStack = new Stack(app, 'OtherAccStack', {
    env: {
        account: repoAcc,
    }
});

const repoName = 'CrossAccountNewRepo';
new cc.Repository(repoAccStack, 'OtherAccRepo', {
    repositoryName: repoName,
});

const repoRole = new iam.Role(repoAccStack, 'OtherAccRole', {
    roleName: 'CrossAccountRole',
    assumedBy: new AccountPrincipal(pipelineAcc),
});
repoRole.addToPolicy(new PolicyStatement()
    .addAllResources()
    .addActions('s3:*', 'codecommit:*', 'kms:*'));
const repoRoleArn = `arn:aws:iam::${repoAcc}:role/CrossAccountRole`;


/*************** the Pipeline Stack *************************/

const pipelineStack = new Stack(app, 'CrossAccountMainStack');

const key = new kms.EncryptionKey(pipelineStack, 'CrossAccountKmsKey');
const artifactBucket = new s3.Bucket(pipelineStack, 'CrossAccountPipelineBucket', {
    bucketName: 'codepipeline-us-west-2-cross-account-code-pipeline-bucket',
    encryptionKey: key,
    encryption: s3.BucketEncryption.Kms,
});
artifactBucket.grantReadWrite(
    new HackyIdentity(new ArnPrincipal(repoRoleArn)));

const pipeline = new cp.Pipeline(pipelineStack, 'CrossAccountPipeline', {
    pipelineName: 'CrossAccountPipeline',
    artifactBucket: artifactBucket,
});
pipeline.addToRolePolicy(new PolicyStatement()
    .addResource(repoRoleArn)
    .addActions('sts:AssumeRole'));

const sourceStage = new cp.Stage(pipeline, 'Source');
const sourceAction = new cp.CodeCommitSource(sourceStage, 'Source', {
    artifactName: 'SourceOutput',
    repository: cc.RepositoryRef.import(pipelineStack, 'ImportedRepo', {
        repositoryName: new codecommit.RepositoryName(repoName)
    }),
    roleArn: repoRoleArn,
});

const buildStage = new cp.Stage(pipeline, 'Build');
const codeBuildProject = new cb.BuildProject(pipelineStack, 'CrossAccountBuildProject', {
    source: new cb.CodePipelineSource(),
    artifacts: new cb.CodePipelineBuildArtifacts(),
    environment: {
        image: 'aws/codebuild/java:openjdk-8',
    },
    encryptionKey: key,
});
new cp.CodeBuildAction(buildStage, 'Build', {
    source: sourceAction,
    project: codeBuildProject,
});

process.stdout.write(app.run());

(this required adding the field roleArn to the CodePipeline Actions, and HackyIdentity is a very simple class:

import * as iam from 'aws-cdk-iam';
import { PolicyPrincipal, PolicyStatement } from 'aws-cdk';

class HackyIdentity implements iam.IIdentityResource {
    public readonly principal: PolicyPrincipal;

    constructor(principal: PolicyPrincipal) {
        this.principal = principal;
    }

    attachManagedPolicy(_: any): void {
    }

    attachInlinePolicy(_: iam.Policy): void {
    }

    addToPolicy(_: PolicyStatement): void {
    }
}

)

As can bee seen, I had to:

• Give physical names to the IAM resources, like the Role in the CodeCommit Account that will be assumed by CodePipeline. This is in order to construct the ARNs needed to then be granted permissions
  (that Role needs access to the Pipeline's S3 bucket, and the CodePipeline will assume it, so needs sts:AssumeRole for it).
• I've actually given the CodeCommit Role * access to S3 - that's because I don't always know what's the ARN of the Bucket, as it can be managed by the L2 CodePipeline Construct (in this example, that's not the case, but in general, that's a problem - that means there's a circular dependency between resources in the 2 accounts). Same remark about the KMS key (that's actually required to make a cross-account CodePipeline work, otherwise the artifacts are encrypted with the default account key, and later stages in the Pipeline won't have access to the source retrieved from CodeCommit).

The dream experience would be something like:

// in the CodeCommit account / stack

const repository = new cc.Repository(repoAccStack, 'OtherAccRepo', {
    repositoryName: 'SomeName',
    // ...
});

// in the CodePipeline account / stack
new cp.CodeCommitSource(sourceStage, 'Source', {
    artifactName: 'SourceOutput',
    repository: repository,
});
// At this moment, the L2 CodePipeline construct notices that the Repository
// passed as the argument when constructing the `CodeCommitSource` is not from the same account as the Pipeline.
// In this case, it add the appropriate Role in the CodeCommit stack,
// adds the permissions for the CodePipeline Role to assume it,
// and makes sure that Role has access to the Pipeline's S3 Bucket,
// as well as the CodeCommit repository itself,
// and then sets it as the Role used in this Action.

I'm not sure how realistic it is to get this experience,
but I believe it's the best one we can provide for this case.

[rix0rrr]

Obligatory mention of #233, which defines a nice and general solution which also encompasses x-stack references.

[rix0rrr]

Yes, you're completely right. I have some cross-stack action going on right now and I'm having to do the same things. Just know that we're thinking about it & working on it! :)

As a general comment: in general we recommend defining a class for every Stack that you want to model. We don't always follow this recommendation ourselves, but only for small demoes and integ tests. Your code will automatically end up more organized with clearer interoperability points between the stacks if you divide them into classes.

[eladb]

@rix0rrr wrote:

Right now, if we're referencing objects between stacks, we can't use Ref and Fn::GetAtt anymore. Consumers need to be constantly cognisant of this, and drop down to physically naming their resources and constructing ARN strings to make these cross-references.

This breaks abstraction because you now need to know (when consuming a resource) whether it's a same-stack or a cross-stack resource.

I propose we build this concept into the framework (potentially at the L2 level): we'll introduce a new type of Token, called StackAwareToken, which will know what object it has been created for, and hence in what Stack that object lives. We'll also propagate the "current stack" while resolving Tokens.

Now, if the current stack != owning stack, the StackToken has the ability to

• Throw a helpful error, because this will never work at CloudFormation deployment time.
• Check whether the owning resource is physically named. If not, throw a helpful error. If it is, return the name or construct the ARN on the spot, whatever the case may be.

Doing this (as opposed to some Output/Export system, potentially involving Toolkit Context Providers) has the advantage that we can avoid ordering between stack deployments, which has two benefits:

• Easier on the user; as long as you eventually deploy everything, the order doesn't matter.
• We can do bidirectional references between stacks (which could never work if we needed to establish a total ordering between stack deploys), which is beneficial for things like IAM permissions.

There is the downside that potential changes in naming aren't automatically propagated/signaled to the user. For Exports, you would be PREVENTED from changing anything that a downstream stack depends on, such as a resource name. (For Outputs or SSM parameters there is no such constraint I think).

[rix0rrr]

The previous comment was written at a time where Stacks were front and center, but everything gets so much better if we automatically slice stacks.

When just build a more generic object graph (#233) and slice it down to stacks, we can do the work to transparently make this work at that point.

[eladb]

Agreed (just captured for posterity), however, even if we automatically slice stacks, there still needs to be a mechanism that resolves these cross-stack references, be them explicit or implicit

[kennu]

Use case: Route53 hostname records in a shared hosted zone

• Stack A -- defines a CloudFront distribution in AWS account A (aaa.example.org)
• Stack B -- defines a CloudFront distribution in AWS account B (bbb.example.org)
• Stack C -- Creates Route53 ALIAS records aaa.example.org and bbb.example.org in a single hosted zone, needs the distribution domain names as input

The challenge is that account A is using a different profile than account B in ~/.aws/credentials so AWS CDK would need to access both profiles to retrieve the distribution domain names.