SGO/README.md

SGO: Security Groups Observatory

A web-based tool for exploring AWS EC2 instances and Security Groups with direct AWS import, MFA support, and CSV export capabilities.

Quick Start

# 1. Create .env file with your AWS credentials path
cat > .env << EOF
AWS_CONFIG_PATH=$HOME/.aws
PUID=$(id -u)
PGID=$(id -g)
EOF

# 2. Start the container
docker-compose up --build
# or with Podman:
podman-compose up --build

# 3. Open browser to http://localhost:5000

# 4. Select AWS profiles, enter MFA codes, and import!

⚠️ Security Warning

This application is designed for LOCAL USE ONLY. Do NOT expose it to the internet.

• SGO has no authentication or authorization mechanisms
• It provides direct access to your AWS infrastructure data
• It reads AWS credentials from your local system
• Exposing it publicly would allow unauthorized access to sensitive AWS information

Always run on localhost (127.0.0.1) only. Never expose port 5000 to external networks.

Features

• Direct AWS Import: Import data directly from AWS using ~/.aws/config with MFA/OTP support
• Parallel Import: Import from multiple AWS accounts simultaneously
• Search & Filter: Search by EC2 name, SG name, instance ID, group ID, or IP address
• Regex Search: Enable regex checkbox for advanced pattern matching
• Filter by Type: View all resources, only EC2 instances, or only Security Groups
• CSV Export: Export search results, EC2 details, SG details, and security group rules to CSV
• Detailed Views:
  • EC2 View: Shows EC2 instance details with nested boxes for attached Security Groups
  • Security Group View: Shows SG details with nested boxes for all attached EC2 instances
• Security Group Rules: View and search ingress/egress rules for any security group
• Statistics Dashboard: Quick overview of total SGs, EC2s, and accounts

Quick Start (Container - Recommended)

The easiest way to run SGO is using Docker or Podman. Works on Linux, macOS, and Windows.

Prerequisites

Install either:

• Docker: https://docs.docker.com/get-docker/
• Podman: https://podman.io/getting-started/installation

Run the Application

# Docker
docker-compose up --build

# Podman
podman-compose up --build

Environment Variables

SGO supports configuration through environment variables. Create a .env file:

# Copy the example file
cp .env.example .env

# Edit with your settings
nano .env  # or your preferred editor

Or create it manually:

cat > .env << EOF
AWS_CONFIG_PATH=$HOME/.aws
PUID=$(id -u)
PGID=$(id -g)
EOF

Available Options:

Variable	Description	Required	Default
AWS_CONFIG_PATH	Absolute path to AWS credentials directory	Yes	None
PUID	User ID for file permissions	No	1000
PGID	Group ID for file permissions	No	1000
DATA_PATH	Path for database storage (local mode)	No	./data
SGO_PORT	Port to expose on host	No	5000
DEBUG	Enable Flask debug logging	No	false
FLASK_ENV	Flask environment	No	production

Data Storage Options

Option 1: Docker Volume (Default - Recommended)

• Data stored in Docker-managed volume sgo-data
• Survives container restarts and rebuilds
• Better performance on macOS/Windows
• Use default docker-compose.yml

Option 2: Local Directory

• Data stored in ./data directory
• Easy to backup and access
• Better for development
• Use docker-compose.local.yml:

docker-compose -f docker-compose.local.yml up --build
# or
podman-compose -f docker-compose.local.yml up --build

Or edit docker-compose.yml and swap the volume configuration as indicated in comments.

User/Group Configuration

To avoid permission issues, set PUID and PGID to match your host user:

# Find your IDs
id -u  # Your PUID
id -g  # Your PGID

# Add to .env file
echo "PUID=$(id -u)" >> .env
echo "PGID=$(id -g)" >> .env

Stopping the Application

# Stop with Ctrl+C, or:
docker-compose down  # Docker
podman-compose down  # Podman

# To also remove the data volume:
docker-compose down -v

Quick Start (Local Python)

If you prefer to run without containers:

1. Install Dependencies

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Start the Application

python app.py

3. Open Browser

Navigate to http://localhost:5000

Important Notes

• ⚠️ LOCAL USE ONLY: Never expose this application to the internet. It has no authentication and provides access to sensitive AWS data.
• Database Persistence: When using containers, the database persists in the ./data directory
• Session Caching: AWS sessions are cached for 55 minutes, allowing multiple refreshes without re-authentication
• Parallel Import: All selected AWS accounts are imported simultaneously for maximum speed

AWS Configuration

MFA Device Setup

For profiles that require MFA, add your MFA device ARN to ~/.aws/config:

[profile nonprod-p1p2-admin]
region = us-west-2
mfa_serial = arn:aws:iam::131340773912:mfa/your-username

Finding Your MFA Device ARN

1. Go to AWS IAM Console
2. Navigate to Users → Your User → Security Credentials
3. Copy the ARN from "Assigned MFA device"

How MFA Works in the GUI

1. The import page shows all profiles from ~/.aws/config
2. Select the profiles you want to import
3. Enter MFA codes in the text boxes (one per profile)
4. Click "Start Import" to begin
5. Real-time progress shows authentication and data fetching
6. MFA session is valid for 1 hour - refresh without re-entering codes during this window

Usage

Search

1. Type in the search box (minimum 2 characters)
2. Results appear instantly as you type
3. Filter by resource type using the buttons: All Resources | EC2 Instances | Security Groups
4. Enable Regex: Check the "Regex" box to use regular expressions
   • Example: ^prod-.*-\d+$ finds names starting with "prod-" and ending with numbers
   • Example: (dev|test|qa) finds names containing dev, test, or qa
   • Example: 10\.0\.\d+\.\d+ finds IP addresses in the 10.0.x.x range

View Details

EC2 Instance View:

• Click on any EC2 instance from search results
• Main card shows EC2 details (Instance ID, IP, State, Account, Tags)
• Nested cards show all attached Security Groups with their details

Security Group View:

• Click on any Security Group from search results
• Main card shows SG details (Group ID, Name, Ingress Rules, Wave, Tags)
• Nested cards show all EC2 instances using this Security Group

View Security Group Rules

When viewing security groups (either attached to an EC2 or directly):

1. Click the View Rules button on any security group card
2. A modal opens showing all ingress and egress rules
3. Switch between Ingress and Egress tabs
4. Use the search box to filter rules by protocol, port, source, or description
5. Rules are displayed in a compact table format with:
   • Protocol (TCP, UDP, ICMP, All)
   • Port Range
   • Source Type (CIDR, Security Group, Prefix List)
   • Source (IP range or SG ID)
   • Description

Navigate

• Click ← Back to Search to return to search results
• Perform a new search at any time
• Click outside the rules modal to close it

Export to CSV

SGO provides comprehensive CSV export capabilities:

Search Results Export:

• Click the 💾 Export button in the view controls (top right)
• Exports all current search results with filters applied
• Includes: Type, Name, ID, Account, State, IP, Security Groups count, Wave, Git info

EC2 Instance Details Export:

• Click the 💾 Export button in any EC2 detail card
• Exports complete EC2 information including:
  • Instance details (ID, name, state, IP, account info)
  • All AWS tags
  • Attached security groups with their details

Security Group Details Export:

• Click the 💾 Export button in any SG detail card
• Exports complete SG information including:
  • Group details (ID, name, wave, rule counts)
  • All AWS tags
  • Attached EC2 instances with their details

Security Group Rules Export:

• Click the 💾 Export button in the rules modal
• Exports all ingress and egress rules with:
  • Rule details (direction, protocol, ports, source)
  • Group ID, account ID
  • Git file and commit information from tags

All exports include timestamps in filenames and proper CSV escaping.

Data Structure

Security Groups Table

• Account ID & Name
• Group ID & Name
• Tag Name
• Wave Tag
• Git Repo Tag
• Ingress Rule Count

EC2 Instances Table

• Account ID & Name
• Instance ID
• Tag Name
• State (running, stopped, etc.)
• Private IP Address
• Security Groups (IDs and Names)
• Git Repo Tag

File Structure

sgo/
├── app.py                      # Flask web application
├── import_from_aws.py          # AWS direct import functions
├── requirements.txt            # Python dependencies
├── Dockerfile                  # Container image definition
├── docker-compose.yml          # Container orchestration (Docker volume)
├── docker-compose.local.yml    # Alternative with local directory storage
├── entrypoint.sh               # Container entrypoint with PUID/PGID support
├── .dockerignore               # Files to exclude from container
├── .env.example                # Example environment configuration
├── .gitignore                  # Git ignore patterns
├── README.md                   # This file
├── data/                       # Local data directory (if using local mode)
│   └── aws_export.db           # SQLite database
├── static/                     # CSS and images
└── templates/                  # HTML

License

This project is dual-licensed:

• FREE for individual, personal, educational, and non-commercial use
• PAID LICENSE REQUIRED for commercial use by businesses and organizations

You may NOT modify this software for the purpose of selling or commercially distributing it.

See the LICENSE file for full details.

For commercial licensing inquiries, please open an issue in this repository.