SGO/README.md

# SGO: Security Groups Observatory

A web-based tool for exploring AWS EC2 instances and Security Groups with direct AWS import, MFA support, and CSV export capabilities.

## Quick Start

```bash
# 1. Create .env file with your AWS credentials path
cat > .env << EOF
AWS_CONFIG_PATH=$HOME/.aws
PUID=$(id -u)
PGID=$(id -g)
EOF

# 2. Start the container
docker-compose up --build
# or with Podman:
podman-compose up --build

# 3. Open browser to http://localhost:5000

# 4. Select AWS profiles, enter MFA codes, and import!
```

## ⚠️ Security Warning

**This application is designed for LOCAL USE ONLY. Do NOT expose it to the internet.**

- SGO has no authentication or authorization mechanisms
- It provides direct access to your AWS infrastructure data
- It reads AWS credentials from your local system
- Exposing it publicly would allow unauthorized access to sensitive AWS information

**Always run on localhost (127.0.0.1) only. Never expose port 5000 to external networks.**

## Features

- **Direct AWS Import**: Import data directly from AWS using `~/.aws/config` with MFA/OTP support
- **Parallel Import**: Import from multiple AWS accounts simultaneously
- **Search & Filter**: Search by EC2 name, SG name, instance ID, group ID, or IP address
- **Regex Search**: Enable regex checkbox for advanced pattern matching
- **Filter by Type**: View all resources, only EC2 instances, or only Security Groups
- **CSV Export**: Export search results, EC2 details, SG details, and security group rules to CSV
- **Detailed Views**:
  - **EC2 View**: Shows EC2 instance details with nested boxes for attached Security Groups
  - **Security Group View**: Shows SG details with nested boxes for all attached EC2 instances
- **Security Group Rules**: View and search ingress/egress rules for any security group
- **Statistics Dashboard**: Quick overview of total SGs, EC2s, and accounts

## Quick Start (Container - Recommended)

The easiest way to run SGO is using Docker or Podman. Works on Linux, macOS, and Windows.

### Prerequisites

Install either:
- **Docker**: https://docs.docker.com/get-docker/
- **Podman**: https://podman.io/getting-started/installation

### Run the Application

```bash
# Docker
docker-compose up --build

# Podman
podman-compose up --build
```

### Environment Variables

SGO supports configuration through environment variables. Create a `.env` file:

```bash
# Copy the example file
cp .env.example .env

# Edit with your settings
nano .env  # or your preferred editor
```

Or create it manually:

```bash
cat > .env << EOF
AWS_CONFIG_PATH=$HOME/.aws
PUID=$(id -u)
PGID=$(id -g)
EOF
```

**Available Options:**

| Variable | Description | Required | Default |
|----------|-------------|----------|---------|
| `AWS_CONFIG_PATH` | Absolute path to AWS credentials directory | Yes | None |
| `PUID` | User ID for file permissions | No | `1000` |
| `PGID` | Group ID for file permissions | No | `1000` |
| `DATA_PATH` | Path for database storage (local mode) | No | `./data` |
| `SGO_PORT` | Port to expose on host | No | `5000` |
| `DEBUG` | Enable Flask debug logging | No | `false` |
| `FLASK_ENV` | Flask environment | No | `production` |

### Data Storage Options

**Option 1: Docker Volume (Default - Recommended)**
- Data stored in Docker-managed volume `sgo-data`
- Survives container restarts and rebuilds
- Better performance on macOS/Windows
- Use default `docker-compose.yml`

**Option 2: Local Directory**
- Data stored in `./data` directory
- Easy to backup and access
- Better for development
- Use `docker-compose.local.yml`:

```bash
docker-compose -f docker-compose.local.yml up --build
# or
podman-compose -f docker-compose.local.yml up --build
```

Or edit `docker-compose.yml` and swap the volume configuration as indicated in comments.

### User/Group Configuration

To avoid permission issues, set `PUID` and `PGID` to match your host user:

```bash
# Find your IDs
id -u  # Your PUID
id -g  # Your PGID

# Add to .env file
echo "PUID=$(id -u)" >> .env
echo "PGID=$(id -g)" >> .env
```

### Stopping the Application

```bash
# Stop with Ctrl+C, or:
docker-compose down  # Docker
podman-compose down  # Podman

# To also remove the data volume:
docker-compose down -v
```

## Quick Start (Local Python)

If you prefer to run without containers:

### 1. Install Dependencies

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```

### 2. Start the Application

```bash
python app.py
```

### 3. Open Browser

Navigate to `http://localhost:5000`

## Important Notes

- **⚠️ LOCAL USE ONLY**: Never expose this application to the internet. It has no authentication and provides access to sensitive AWS data.
- **Database Persistence**: When using containers, the database persists in the `./data` directory
- **Session Caching**: AWS sessions are cached for 55 minutes, allowing multiple refreshes without re-authentication
- **Parallel Import**: All selected AWS accounts are imported simultaneously for maximum speed


## AWS Configuration

### MFA Device Setup

For profiles that require MFA, add your MFA device ARN to `~/.aws/config`:

```ini
[profile nonprod-p1p2-admin]
region = us-west-2
mfa_serial = arn:aws:iam::131340773912:mfa/your-username
```

### Finding Your MFA Device ARN

1. Go to AWS IAM Console
2. Navigate to Users → Your User → Security Credentials
3. Copy the ARN from "Assigned MFA device"

### How MFA Works in the GUI

1. The import page shows all profiles from `~/.aws/config`
2. Select the profiles you want to import
3. Enter MFA codes in the text boxes (one per profile)
4. Click "Start Import" to begin
5. Real-time progress shows authentication and data fetching
6. MFA session is valid for 1 hour - refresh without re-entering codes during this window

## Usage

### Search

1. Type in the search box (minimum 2 characters)
2. Results appear instantly as you type
3. Filter by resource type using the buttons: **All Resources** | **EC2 Instances** | **Security Groups**
4. **Enable Regex**: Check the "Regex" box to use regular expressions
   - Example: `^prod-.*-\d+$` finds names starting with "prod-" and ending with numbers
   - Example: `(dev|test|qa)` finds names containing dev, test, or qa
   - Example: `10\.0\.\d+\.\d+` finds IP addresses in the 10.0.x.x range

### View Details

**EC2 Instance View:**
- Click on any EC2 instance from search results
- Main card shows EC2 details (Instance ID, IP, State, Account, Tags)
- Nested cards show all attached Security Groups with their details

**Security Group View:**
- Click on any Security Group from search results
- Main card shows SG details (Group ID, Name, Ingress Rules, Wave, Tags)
- Nested cards show all EC2 instances using this Security Group

### View Security Group Rules

When viewing security groups (either attached to an EC2 or directly):
1. Click the **View Rules** button on any security group card
2. A modal opens showing all ingress and egress rules
3. Switch between **Ingress** and **Egress** tabs
4. Use the search box to filter rules by protocol, port, source, or description
5. Rules are displayed in a compact table format with:
   - Protocol (TCP, UDP, ICMP, All)
   - Port Range
   - Source Type (CIDR, Security Group, Prefix List)
   - Source (IP range or SG ID)
   - Description

### Navigate

- Click **← Back to Search** to return to search results
- Perform a new search at any time
- Click outside the rules modal to close it

### Export to CSV

SGO provides comprehensive CSV export capabilities:

**Search Results Export:**
- Click the **💾 Export** button in the view controls (top right)
- Exports all current search results with filters applied
- Includes: Type, Name, ID, Account, State, IP, Security Groups count, Wave, Git info

**EC2 Instance Details Export:**
- Click the **💾 Export** button in any EC2 detail card
- Exports complete EC2 information including:
  - Instance details (ID, name, state, IP, account info)
  - All AWS tags
  - Attached security groups with their details

**Security Group Details Export:**
- Click the **💾 Export** button in any SG detail card
- Exports complete SG information including:
  - Group details (ID, name, wave, rule counts)
  - All AWS tags
  - Attached EC2 instances with their details

**Security Group Rules Export:**
- Click the **💾 Export** button in the rules modal
- Exports all ingress and egress rules with:
  - Rule details (direction, protocol, ports, source)
  - Group ID, account ID
  - Git file and commit information from tags

All exports include timestamps in filenames and proper CSV escaping.

## Data Structure

### Security Groups Table
- Account ID & Name
- Group ID & Name
- Tag Name
- Wave Tag
- Git Repo Tag
- Ingress Rule Count

### EC2 Instances Table
- Account ID & Name
- Instance ID
- Tag Name
- State (running, stopped, etc.)
- Private IP Address
- Security Groups (IDs and Names)
- Git Repo Tag

## File Structure

```
sgo/
├── app.py                      # Flask web application
├── import_from_aws.py          # AWS direct import functions
├── requirements.txt            # Python dependencies
├── Dockerfile                  # Container image definition
├── docker-compose.yml          # Container orchestration (Docker volume)
├── docker-compose.local.yml    # Alternative with local directory storage
├── entrypoint.sh               # Container entrypoint with PUID/PGID support
├── .dockerignore               # Files to exclude from container
├── .env.example                # Example environment configuration
├── .gitignore                  # Git ignore patterns
├── README.md                   # This file
├── data/                       # Local data directory (if using local mode)
│   └── aws_export.db           # SQLite database
├── static/                     # CSS and images
└── templates/                  # HTML
```


## License

This project is dual-licensed:

- **FREE** for individual, personal, educational, and non-commercial use
- **PAID LICENSE REQUIRED** for commercial use by businesses and organizations

You may NOT modify this software for the purpose of selling or commercially distributing it.

See the [LICENSE](LICENSE) file for full details.

For commercial licensing inquiries, please open an issue in this repository.