Feature/aws container services#1719
Open
philipwubbleyou wants to merge 6 commits intonccgroup:masterfrom
Open
Conversation
- Remove duplicate service initialization in services.py (lines 136-145) ECR, ECS, EKS were being initialized twice, causing conflicts - Fix ECR facade async/await issues in ecr.py: * Added missing await in get_images() method (line 42) * Rewrote _get_image() to properly handle async operations * Fixed inefficient loop that was fetching ALL repositories for each image - Fix ECS cluster parsing in clusters.py: * Added proper handling for missing containerInsights setting * Added default value to prevent KeyError * Added safety checks with .get() methods - Fix facade initialization in base.py: * Moved ECR/ECS/EKS to standard service initialization * Removed from proprietary services section since they're now open-source These fixes address critical bugs that would cause runtime errors and performance issues in production environments.
Fixed critical bugs in ECS facade that caused API errors: 1. Service ARN parsing (_get_service): - Was incorrectly listing ALL clusters and concatenating ARNs - Now properly extracts cluster name from service ARN - Format: arn:aws:ecs:region:account:service/cluster-name/service-name 2. Task ARN parsing (_get_tasks): - Same issue as services - was concatenating all cluster ARNs - Now properly extracts cluster name from task ARN - Format: arn:aws:ecs:region:account:task/cluster-name/task-id 3. Missing await statements: - Added await to get_services() list_services call - Added await to get_tasks() list_tasks call - Added await to _get_service() describe_services call - Added await to _get_tasks() describe_tasks call These fixes resolve the "Unsupported resource type: cluster" errors that occurred when scanning accounts with ECS services and tasks. Tested with AWS account containing multiple ECS clusters, services, and tasks. All resources now scan successfully.
Fixed critical bugs where code assumed all fields exist in API responses: 1. ECS Tasks (tasks.py): - containerInstanceArn: Only exists for EC2 launch type, not Fargate - All fields now use .get() with sensible defaults - Added safe container parsing with existence checks 2. ECS Services (services.py): - launchType: Optional when using capacity provider strategies - All fields now use .get() with sensible defaults - Added safe deployment parsing with existence checks These fixes handle the different ECS deployment models: - Fargate vs EC2 launch types - Capacity provider strategies vs direct launch types - Services with/without active deployments - Tasks with/without containers Resolves KeyError: 'containerInstanceArn' and KeyError: 'launchType' errors when scanning ECS resources.
Fixed critical bugs in RDS snapshot attribute fetching that caused 132 RDS snapshots to fail and not be counted as resources. Issues fixed: 1. TypeError: "argument of type 'ClientError' is not iterable" - Line 119: Was checking if string 'in' ClientError object - Fixed: Convert exception to string first with str(e) 2. Improved throttling error handling: - AWS rate limits are hit when scanning many snapshots - Now properly detects and handles throttling gracefully - Uses print_warning for rate limits instead of print_exception 3. Applied same fixes to cluster snapshot attributes function Root cause analysis: - The official version scans faster (no ECS/ECR) so hits rate limits less - Our fork scans ECS/ECR first, using API quota, then RDS hits limits - The TypeError prevented proper error handling, causing silent failures - Snapshots without 'Attributes' key couldn't be parsed, reducing resource count This should restore the full 486 RDS resources in scan results.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds support for scanning AWS container services (ECS, ECR, EKS) and includes critical bug fixes discovered during testing and integration.
Fixes #1491
Based on the excellent work by @kedar1704 in #1587
Type of change
Summary of Changes
New Services Added
Bug Fixes
1. ECS Service/Task ARN Parsing (
ScoutSuite/providers/aws/facade/ecs.py)Problem: Code attempted to determine cluster ownership by listing ALL clusters and concatenating ARNs with
"".join(), resulting in malformed cluster identifiers.Error:
InvalidParameterException: Unsupported resource type: clusterFix: Extract cluster name directly from ARN structure:
arn:aws:ecs:region:account:service/cluster-name/service-namearn:aws:ecs:region:account:task/cluster-name/task-idcluster_name = arn.split('/')[1]2. ECS Optional Field Handling
Problem: Code assumed all tasks have
containerInstanceArnand all services havelaunchType, causing KeyError exceptions.Root Cause:
containerInstanceArn(only present for EC2 launch type)launchTypeFix: Changed all field access to use
.get()with appropriate defaultsFiles:
ScoutSuite/providers/aws/resources/ecs/tasks.pyScoutSuite/providers/aws/resources/ecs/services.py3. RDS Snapshot Error Handling (
ScoutSuite/providers/aws/facade/rds.py)Problem: TypeError when checking exception type:
Fix:
Also added proper handling for AWS Throttling errors with warning-level logging instead of exceptions.
4. Service Initialization Conflicts
Problem: ECR/ECS/EKS were initialized twice (standard + proprietary), causing resource conflicts
Fix: Removed duplicate initialization from proprietary services section
Files:
ScoutSuite/providers/aws/services.pyScoutSuite/providers/aws/facade/base.py5. Missing Async/Await Keywords
Problem: Several async operations lacked
await, causing synchronous executionFix: Added
awaitto all async calls in ECS/ECR facadesFiles:
ScoutSuite/providers/aws/facade/ecs.pyScoutSuite/providers/aws/facade/ecr.pySecurity Rules Added
ECR (2 rules)
ecr-image-scanning-enabled: Verifies image scanning on push is enabledecr-tag-immutability-enabled: Checks tag immutability configurationECS (1 rule)
ecs-container-insights-enabled: Validates Container Insights is enabled for monitoringEKS (2 rules)
eks-cluster-logging-enabled: Ensures cluster logging is configuredeks-endpoint-private-access-enable: Validates private endpoint accessAll rules include CIS compliance mappings and remediation guidance.
Testing Performed
Tested against multiple AWS accounts containing:
Verified:
Files Changed
New Files (Facades)
ScoutSuite/providers/aws/facade/ecr.pyScoutSuite/providers/aws/facade/ecs.pyScoutSuite/providers/aws/facade/eks.pyNew Files (Resources)
ScoutSuite/providers/aws/resources/ecr/(base.py, images.py, repositories.py)ScoutSuite/providers/aws/resources/ecs/(base.py, clusters.py, services.py, tasks.py)ScoutSuite/providers/aws/resources/eks/(base.py, clusters.py, nodegroups.py)New Files (Rules)
ScoutSuite/providers/aws/rules/findings/ecr-image-scanning-enabled.jsonScoutSuite/providers/aws/rules/findings/ecr-tag-immutablity-enabled.jsonScoutSuite/providers/aws/rules/findings/ecs-container-insights-enabled.jsonScoutSuite/providers/aws/rules/findings/eks-cluster-logging-enabled.jsonScoutSuite/providers/aws/rules/findings/eks-endpoint-private-access-enable.jsonNew Files (HTML Templates)
ScoutSuite/output/data/html/partials/aws/services.ecr.regions.id.images.htmlScoutSuite/output/data/html/partials/aws/services.ecr.regions.id.repositories.htmlScoutSuite/output/data/html/partials/aws/services.ecs.regions.htmlScoutSuite/output/data/html/partials/aws/services.ecs.regions.id.clusters.htmlScoutSuite/output/data/html/partials/aws/services.ecs.regions.id.services.htmlScoutSuite/output/data/html/partials/aws/services.ecs.regions.id.tasks.htmlScoutSuite/output/data/html/partials/aws/services.eks.regions.id.clusters.htmlScoutSuite/output/data/html/partials/aws/services.eks.regions.id.nodegroups.htmlModified Files
ScoutSuite/providers/aws/facade/base.py(added container service facades)ScoutSuite/providers/aws/facade/rds.py(bug fixes only)ScoutSuite/providers/aws/services.py(added container service initialization)ScoutSuite/providers/aws/metadata.json(added service metadata)ScoutSuite/providers/aws/rules/rulesets/default.json(registered new rules)Compatibility & Breaking Changes
--skip ecs ecr eksif neededKnown Behavior
Under high API load, AWS may rate-limit RDS snapshot attribute fetching. This is handled gracefully with warning-level logging. Snapshots are still discovered and included in resource counts; only detailed attributes may be incomplete. This affects <1% of resources in typical scenarios.
Checklist:
Acknowledgments