AWS Security Configuration Scanner

The security scanner project has now been suspended. If you want to scan your workloads, I suggest you look at prowler.

Large enterprises tend to invest into CSPM systems (Cloud Security Posture Management) like Dome9, PrismaCloud, or Orca Security. For smaller companies, it may be cost prohibitive to invest in a CSPM, so they tend to simply do nothing , and hope they don't have any breaches. This is a dangerous place to be in.

Let's assume you do look at tools like Trusted Advisor once in a while. It will show you where some of the big ticket items are that you need to look at, but it doesn't go into a lot of detail. That's where the AWS Security Info Configuration Scanner comes in. The AWS Security scanner is a Python project I've been working on for the past 2 years, and it is finally ready for release.

As the title implies, it's a tool you can use to scan the configuration of your AWS account. It has a number of built-in security controls that will give you an overview of where the security issues in your AWS account could be. At a high level, the AWS CIS Foundations Benchmark was used as the basis for the majority of security controls.

I can already hear some of you saying : Why should I use this script, when I can simply use Security Hub? And you'd be right — you could use Security Hub (and in fact, I highly recommend it!). The big difference is that with Security Hub, you'll have Config rules setup, and Config will incur additional charges. This is not necessarily a bad thing. The problem however, is that Security Hub will keep generating alerts, and unless you're actively monitoring them, the alerts will simply go into a blackhole, never to be seen again.

Why use this script then? I view it more like an audit tool. It has the ability to generate a point-in-time snapshot of what the security configuration of your AWS looks like, and the output can then be used by auditors to discuss and challenge the findings with the various cloud security architecture teams.

How to use it

I would recommend that you run the script from the us-east-1 region. Since this is the central region for AWS (where all the IAM function live), most of the API calls will occur against this region, so it's recommended that you either use the Shell, or a Spot instance in that region to run the script.

Shell

Fire up the CloudShell in the us-east-1 region. Assuming you have ReadOnly access to the AWS account, simply execute the following lines of code

    git clone http://github.com/massyn/aws-security
    pip3 install boto3 mako --upgrade
    python3 aws-security/scanner/scanner.py --collect /tmp/%a-%d.json --report /tmp/%a.html

That's it! The script should start running. Depending on the size of your environment, it may take about 30 minutes to run, maybe more.

security-scanner-1.png

Spot instance

This is a work in progress. I have been successful in running a spot instance to execute the script. I am busy packaging the solution, and will update this blog post once it is ready. Essentially, you need to :

Operation

The script connects to AWS using the default credentials, and starts to interrogate each of the services to retrieve the data. This is where the json files comes in. When it's done with the data extraction, you'll have a single json file that contains (most) of the system configuration that has been defined on your AWS account. This has huge implications. If you're interested in digging through the config, you will be able to generate your own jmespath queries to retrieve anything your heart desires.

Once the json file has been created, the policy parser kicks in. It will read through the json file, looking for the logic that has been predefined in the script, and then generating a report (in HTML format) of all the findings.

Hidden features

When specifying the output file names (--collect, --report, --evidence), you can specify %a (for the accountId) or %d for the date. This allows you to have a batch file or a shell script you can run against a number of accounts, and it will keep a file per account, per day.

You can also request the cloud team to run the json extract for you. Once you have the json file, you can parse the output yourself, using the --nocollect function. This will simply skip the data ingestion function, and read the provided -–collect file, and parse the security rules.

Did you know you can specify an S3 path for the html or json files? That's right! You can store the HTML file directly to S3!

Known issues

What's next?

This is where you come in. The main driver for this project is to give something back to the AWS community, to make AWS a more secure environment for its customers. Some of the things I'd like to still do are:

Community Support

The project is hosted in GitHub, and being in GitHub means that you can fork your own copy of the code, and adjust it. All I ask is that you give credit, and that you contribute to the overall project with source code suggestions, or new policies you'd like to see.

Reference guide

collector

The collector script is responsible to connect to AWS, and retrieve the data objects from the internal database.

--collect

Specify the output path where the target file should be stored. Note that throughout execution, the file will be written. This is done by design, that should the script fail, you're able to restart it, without losing any of the previously extracted data.

--nocollect

Skip the data collection part. Useful if you only want to parse the output against a set of policies.

--evidence

Once the policy engine has compared all the policies, you can (optionally) save the output file as a local json file.

--report

The output report (in HTML) can also be stored for consumption through a regular web browser.

Ways to authenticate

    --aws_access_key_id AWS_ACCESS_KEY_ID
    --aws_secret_access_key AWS_SECRET_ACCESS_KEY
    --aws_session_token AWS_SESSION_TOKEN

Release Notes - December 2021

The December release is a significant rewrite. If you relied on the original data format, you'll notice that it is no longer the same.

Release Notes - August 2021

It's been a bit of a quiet month for updates to the AWS Security Info modules. There's been a couple of changes that I'm publishing today.

New features

Bug fixes

Data collection

Policy updates