Skip to main content

Analyze VPC Flow Logs

Cloud Platform live-1 VPC is connected to various MoJ AWS accounts VPCs using Transit Gateway. In order to debug network connectivity issues that are coming to/from Cloud Platform VPC, the VPC flow logs are enabled. These logs are stored in an S3 bucket and can be analyzed using Athena when needed.

The steps involve a pointy-clicky exercise to generate Athena integration by creating a CloudFormation stack using a template and use that as a datasource in Athena.

Follow the steps as mentioned in the blog: AWS blog - Analyze VPC Flow Logs with point-and-click Amazon Athena integration.

Things to note:

  • When creating a CloudFormation template, use the Partition: Daily option. This will create a partition for each day and will make it easier to query the logs for a specific day.
  • Provide Partition start date and end date closer when the issue was observed. The duration is limited to 20 days
  • Create two s3 buckets:
    • one for CloudFormation template
    • one to store the Athena query results which you will provide when creating the CloudFormation stack.
  • When the flow logs are downloaded and the user has investigated the issue, delete the created s3 buckets for CloudFormation template and Athena query results and the CloudFormation stack.
This page was last reviewed on 18 December 2024. It needs to be reviewed again on 18 June 2025 by the page owner #cloud-platform .
This page was set to be reviewed before 18 June 2025 by the page owner #cloud-platform. This might mean the content is out of date.