wiki

wiki/AWS-Configuration.md

@@ -0,0 +1,225 @@
+# AWS Configuration
+
+## Prerequisites
+
+Before using SGO, ensure you have:
+
+1. AWS CLI configured with credentials
+2. Appropriate IAM permissions
+3. MFA device configured (if required by your profiles)
+
+## AWS Profiles Setup
+
+SGO reads profiles from `~/.aws/config`. Ensure your AWS configuration files are set up correctly.
+
+### Basic Profile Configuration
+
+```ini
+[profile my-aws-account]
+region = us-west-2
+```
+
+### Profile with MFA
+
+For profiles that require MFA authentication:
+
+```ini
+[profile nonprod-p1p2-admin]
+region = us-west-2
+mfa_serial = arn:aws:iam::131340773912:mfa/your-username
+```
+
+### Multiple Profiles
+
+You can have multiple profiles in your config file:
+
+```ini
+[default]
+region = us-east-1
+
+[profile production]
+region = us-west-2
+mfa_serial = arn:aws:iam::123456789012:mfa/john.doe
+
+[profile development]
+region = us-west-2
+
+[profile staging]
+region = us-east-1
+mfa_serial = arn:aws:iam::987654321098:mfa/john.doe
+```
+
+## MFA Device Setup
+
+### Finding Your MFA Device ARN
+
+1. Go to AWS IAM Console
+2. Navigate to **Users** → **Your User** → **Security Credentials**
+3. Scroll to **Multi-factor authentication (MFA)**
+4. Copy the ARN from "Assigned MFA device"
+
+Example ARN format:
+```
+arn:aws:iam::123456789012:mfa/username
+```
+
+### Adding MFA to Profile
+
+Add the `mfa_serial` line to your profile in `~/.aws/config`:
+
+```ini
+[profile my-profile]
+region = us-west-2
+mfa_serial = arn:aws:iam::123456789012:mfa/username
+```
+
+## How MFA Works in SGO
+
+1. The import page shows all profiles from `~/.aws/config`
+2. Profiles with `mfa_serial` configured will show an MFA input field
+3. Profiles without `mfa_serial` can import without entering a code
+4. Enter your current MFA/TOTP code (6 digits) for profiles that require it
+5. Click "Start Import" to begin authentication and data import
+6. MFA session is valid for 1 hour
+7. During the session window (55 minutes), you can refresh without re-entering codes
+
+### MFA Code Sources
+
+You can get MFA codes from:
+- Authenticator apps (Google Authenticator, Microsoft Authenticator, Authy, etc.)
+- Hardware MFA devices
+- SMS (if configured)
+
+**Note**: MFA codes expire every 30 seconds, so enter them promptly.
+
+## Required IAM Permissions
+
+Your AWS user/role needs the following permissions to use SGO:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "ec2:DescribeInstances",
+        "ec2:DescribeSecurityGroups",
+        "iam:ListAccountAliases",
+        "sts:GetCallerIdentity"
+      ],
+      "Resource": "*"
+    }
+  ]
+}
+```
+
+### Permission Breakdown
+
+- `ec2:DescribeInstances` - List and describe EC2 instances
+- `ec2:DescribeSecurityGroups` - List and describe security groups
+- `iam:ListAccountAliases` - Get friendly account names
+- `sts:GetCallerIdentity` - Get account ID
+
+## AWS Credentials Location
+
+### Default Location
+
+SGO expects AWS credentials at:
+- Linux/macOS: `~/.aws/`
+- Windows: `%USERPROFILE%\.aws\`
+
+### Custom Location
+
+If your AWS credentials are in a non-standard location, specify it in your `.env` file:
+
+```bash
+AWS_CONFIG_PATH=/path/to/custom/.aws
+```
+
+### Required Files
+
+Ensure these files exist in your AWS credentials directory:
+
+1. **`config`** - Contains profile configurations
+   ```ini
+   [profile my-profile]
+   region = us-west-2
+   mfa_serial = arn:aws:iam::123456789012:mfa/username
+   ```
+
+2. **`credentials`** - Contains access keys
+   ```ini
+   [my-profile]
+   aws_access_key_id = AKIAIOSFODNN7EXAMPLE
+   aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+   ```
+
+## Testing Your Configuration
+
+### Verify AWS CLI Access
+
+```bash
+# Test default profile
+aws sts get-caller-identity
+
+# Test specific profile
+aws sts get-caller-identity --profile my-profile
+
+# Test with MFA
+aws sts get-caller-identity --profile my-profile
+# (will prompt for MFA if configured)
+```
+
+### Verify Permissions
+
+```bash
+# Test EC2 access
+aws ec2 describe-instances --profile my-profile --max-results 1
+
+# Test security groups access
+aws ec2 describe-security-groups --profile my-profile --max-results 1
+```
+
+## Common Configuration Issues
+
+### No Profiles Found
+
+**Problem**: Import page shows "No AWS profiles found"
+
+**Solution**:
+- Verify `~/.aws/config` exists and contains profiles
+- Check file permissions (should be readable)
+- Ensure profiles are properly formatted in config file
+
+### MFA Authentication Fails
+
+**Problem**: "MFA authentication failed" error
+
+**Solution**:
+- Verify MFA code is current (not expired)
+- Check `mfa_serial` is correct in `~/.aws/config`
+- Ensure AWS credentials in `~/.aws/credentials` are valid
+- Try generating a new MFA code
+
+### Permission Denied
+
+**Problem**: "Access Denied" or "Unauthorized" errors
+
+**Solution**:
+- Verify your IAM user/role has required permissions
+- Check if your credentials have expired
+- Ensure you're using the correct profile
+
+### Wrong Region
+
+**Problem**: Not seeing resources you expect
+
+**Solution**:
+- Verify the `region` setting in your profile
+- Remember: EC2 resources are region-specific
+- Try setting the region explicitly:
+  ```ini
+  [profile my-profile]
+  region = us-west-2
+  ```

wiki/Configuration.md

@@ -0,0 +1,217 @@
+# Configuration
+
+## Environment Variables
+
+SGO supports configuration through environment variables. Create a `.env` file in the same directory as your `docker-compose.yml` file.
+
+### Available Options
+
+| Variable | Description | Required | Default |
+|----------|-------------|----------|---------|
+| `AWS_CONFIG_PATH` | Absolute path to AWS credentials directory | Yes | None |
+| `PUID` | User ID for file permissions | No | `1000` |
+| `PGID` | Group ID for file permissions | No | `1000` |
+| `DATA_PATH` | Path for database storage (local mode) | No | `./data` |
+| `SGO_PORT` | Port to expose on host | No | `5000` |
+| `DEBUG` | Enable Flask debug logging | No | `false` |
+| `FLASK_ENV` | Flask environment | No | `production` |
+
+## Data Storage Options
+
+### Option 1: Docker Volume (Default - Recommended)
+
+- Data stored in Docker-managed volume `sgo-data`
+- Survives container restarts and rebuilds
+- Better performance on macOS/Windows
+- Use default `docker-compose.yml`
+
+```bash
+docker-compose up --build
+```
+
+### Option 2: Local Directory
+
+- Data stored in `./data` directory
+- Easy to backup and access
+- Better for development
+- Use `docker-compose.local.yml`:
+
+```bash
+docker-compose -f docker-compose.local.yml up --build
+# or
+podman-compose -f docker-compose.local.yml up --build
+```
+
+Or edit `docker-compose.yml` and swap the volume configuration as indicated in comments.
+
+## User/Group Configuration
+
+To avoid permission issues, set `PUID` and `PGID` to match your host user:
+
+```bash
+# Find your IDs
+id -u  # Your PUID
+id -g  # Your PGID
+
+# Add to .env file
+echo "PUID=$(id -u)" >> .env
+echo "PGID=$(id -g)" >> .env
+```
+
+## Configuration Examples
+
+### Example 1: Basic Setup (Default)
+
+Minimal configuration with Docker volume:
+
+```bash
+# Create .env file
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+EOF
+
+# Run
+docker-compose up --build
+# or: podman-compose up --build
+```
+
+### Example 2: Local Data Directory
+
+Store database in local directory for easy access:
+
+```bash
+# Create .env file
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+DATA_PATH=./data
+EOF
+
+# Use the local compose file
+docker-compose -f docker-compose.local.yml up --build
+```
+
+### Example 3: Custom AWS Credentials Location
+
+If your AWS credentials are in a non-standard location:
+
+```bash
+# Create .env file
+cat > .env << EOF
+AWS_CONFIG_PATH=/path/to/custom/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+EOF
+
+docker-compose up --build
+```
+
+### Example 4: Debug Mode
+
+Enable detailed logging for troubleshooting:
+
+```bash
+# Create .env file
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+DEBUG=true
+EOF
+
+docker-compose up --build
+```
+
+### Example 5: Custom Port
+
+Run on a different port (e.g., 8080):
+
+```bash
+# Create .env file
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+SGO_PORT=8080
+EOF
+
+# Access at http://localhost:8080
+docker-compose up --build
+```
+
+## Quick Reference Commands
+
+### Essential Commands
+
+```bash
+# Start
+docker-compose up --build
+# or: podman-compose up --build
+
+# Stop
+docker-compose down
+# or: Ctrl+C
+
+# View logs
+docker-compose logs -f
+
+# Rebuild after changes
+docker-compose up --build
+
+# Remove everything including data
+docker-compose down -v
+```
+
+### Quick .env Setup
+
+```bash
+# Minimal configuration for most users
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+EOF
+
+# Then run
+docker-compose up --build
+```
+
+### Data Location
+
+**Docker volume (default)**: Managed by Docker, survives rebuilds
+
+```bash
+# Inspect volume
+docker volume inspect sgo-data
+
+# Backup volume
+docker run --rm -v sgo-data:/data -v $(pwd):/backup alpine tar czf /backup/sgo-backup.tar.gz -C /data .
+```
+
+**Local directory**: `./data/aws_export.db`
+
+```bash
+# Use local mode
+docker-compose -f docker-compose.local.yml up --build
+```
+
+## Platform-Specific Notes
+
+### Windows
+
+- Use Git Bash, WSL2, or PowerShell for running scripts
+- Docker Desktop must be running
+- WSL2 backend recommended for better performance
+
+### macOS
+
+- Docker Desktop must be running
+- Ensure Docker has access to your home directory in Docker Desktop settings
+
+### Linux
+
+- You may need to run Docker commands with `sudo` or add your user to the `docker` group
+- Podman works without root by default

wiki/FAQ.md

@@ -0,0 +1,330 @@
+# Frequently Asked Questions (FAQ)
+
+## General Questions
+
+### What is SGO?
+
+SGO (Security Groups Observatory) is a web-based tool for exploring and analyzing AWS EC2 instances and Security Groups. It provides search, filtering, detailed views, and CSV export capabilities for AWS infrastructure.
+
+### Is SGO free to use?
+
+Yes, for personal, educational, and non-commercial use. Commercial use by businesses requires a paid license. See [LICENSE](../LICENSE) for details.
+
+### Can I use SGO in production?
+
+**No, not recommended.** SGO is designed for local use only and has no authentication mechanisms. It should never be exposed to the internet or production networks.
+
+### What AWS services does SGO support?
+
+Currently, SGO supports:
+- EC2 Instances
+- Security Groups (with ingress/egress rules)
+
+Future versions may add support for additional AWS services.
+
+## Installation & Setup
+
+### Do I need Docker or Podman?
+
+Not required, but strongly recommended. You can run SGO with local Python, but containers provide easier setup and better isolation.
+
+### What are PUID and PGID?
+
+PUID (Process User ID) and PGID (Process Group ID) ensure files created by the container have correct ownership on your host system. Set them to your user's UID/GID to avoid permission issues.
+
+```bash
+# Find your values
+id -u  # PUID
+id -g  # PGID
+```
+
+### Can I use Docker volume or local directory for data?
+
+Both are supported:
+- **Docker volume** (default): Better for macOS/Windows, managed by Docker
+- **Local directory**: Better for development, easy to backup
+
+See [Configuration](Configuration.md#data-storage-options) for details.
+
+### Where are my AWS credentials stored?
+
+AWS credentials are NOT stored by SGO. The application reads from your existing `~/.aws/` directory and only keeps temporary session tokens in memory (cached for 55 minutes).
+
+## AWS & Authentication
+
+### Do I need MFA?
+
+Only if your AWS profile requires it. If your `~/.aws/config` has `mfa_serial` configured for a profile, you'll need to provide an MFA code when importing.
+
+### How long does MFA session last?
+
+AWS MFA sessions last 1 hour. SGO caches them for 55 minutes, allowing you to refresh data multiple times without re-entering MFA codes.
+
+### Can I import multiple AWS accounts?
+
+Yes! SGO supports multiple profiles and can import them in parallel. Select all desired profiles and start imports simultaneously for maximum efficiency.
+
+### What AWS permissions do I need?
+
+Minimum required permissions:
+- `ec2:DescribeInstances`
+- `ec2:DescribeSecurityGroups`
+- `iam:ListAccountAliases`
+- `sts:GetCallerIdentity`
+
+See [AWS Configuration](AWS-Configuration.md#required-iam-permissions) for details.
+
+### Why can't I see my resources?
+
+Common causes:
+1. **Wrong region**: EC2 resources are region-specific
+2. **No permission**: Verify IAM permissions
+3. **Import failed**: Check import progress logs for errors
+4. **Different account**: Ensure you're importing the correct profile
+
+## Features & Usage
+
+### How do I search for resources?
+
+Type in the search box (minimum 2 characters). Search works across:
+- EC2 instance names and IDs
+- Security group names and IDs
+- IP addresses
+- Tags
+
+Enable "Regex" checkbox for pattern matching.
+
+### What's the difference between EC2 view and SG view?
+
+- **EC2 view**: Shows instance details with nested Security Groups
+- **SG view**: Shows security group details with nested EC2 instances
+
+Both views are interactive - click any item to see its details.
+
+### Can I export my data?
+
+Yes! SGO provides CSV export at multiple levels:
+- Search results
+- Individual EC2 instance details
+- Individual Security Group details
+- Security group rules
+
+All exports include proper CSV formatting and timestamps.
+
+### Does SGO modify my AWS resources?
+
+**No.** SGO is read-only and never modifies AWS resources. It only uses `Describe*` and `List*` API calls.
+
+## Performance
+
+### How fast is the import?
+
+Import speed depends on:
+- Number of resources in your AWS account
+- Network latency to AWS
+- Number of accounts being imported
+
+Parallel import significantly reduces total time by importing multiple accounts simultaneously.
+
+### Why is search slow with many resources?
+
+Search is client-side (in your browser) for instant results. With thousands of resources, this can be slower. Tips:
+- Use more specific search terms
+- Use type filters (EC2 only, SG only)
+- Disable regex when not needed
+
+### Can I improve performance?
+
+Yes:
+- **Use Docker volume** instead of local directory (better I/O)
+- **Import selectively**: Only import accounts you need
+- **Use specific searches**: Narrow down results
+- **Use type filters**: Filter by EC2 or SG only
+
+## Data & Privacy
+
+### Is my data sent anywhere?
+
+**No.** SGO runs entirely locally. Data flows:
+1. AWS → Your local SGO instance
+2. Stored in local SQLite database
+3. Never leaves your machine
+
+### Can others see my AWS data?
+
+Only if they have access to:
+- Your computer while SGO is running
+- Your SQLite database file
+- Your Docker volume
+
+**Never expose SGO to the internet.**
+
+### How is data stored?
+
+Data is stored in a local SQLite database:
+- Default location: Docker volume `sgo-data`
+- Alternative: `./data/aws_export.db` (local mode)
+
+### Can I backup my data?
+
+Yes:
+
+**Docker volume backup**:
+```bash
+docker run --rm -v sgo-data:/data -v $(pwd):/backup alpine tar czf /backup/sgo-backup.tar.gz -C /data .
+```
+
+**Local directory backup**:
+```bash
+cp ./data/aws_export.db ./backups/
+```
+
+### How do I clear all data?
+
+**Remove database and start fresh**:
+```bash
+# Docker volume mode
+docker-compose down -v
+
+# Local directory mode
+rm -rf ./data
+```
+
+## Troubleshooting
+
+### Import fails silently
+
+Check the import progress logs in the UI. Common issues:
+- Invalid AWS credentials
+- Expired MFA code
+- Missing IAM permissions
+- Network connectivity
+
+### Container won't start
+
+Common causes:
+1. Port 5000 already in use
+2. Docker/Podman not running
+3. Missing `.env` file
+4. Invalid AWS_CONFIG_PATH
+
+See [Troubleshooting](Troubleshooting.md) for detailed solutions.
+
+### AWS profiles not showing up
+
+Ensure:
+- `~/.aws/config` exists and is readable
+- Profiles are properly formatted
+- `AWS_CONFIG_PATH` in `.env` is correct
+- Container has been restarted after config changes
+
+## Development & Customization
+
+### Can I modify SGO?
+
+Yes, under the license terms:
+- ✅ Modify for personal use
+- ✅ Contribute improvements back
+- ❌ Modify to sell or redistribute commercially
+
+### Can I contribute to SGO?
+
+Yes! Contributions are welcome. See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
+
+### Does SGO support other clouds?
+
+Not currently. SGO is AWS-specific. Support for other cloud providers may be added in the future.
+
+### Can I add custom tags or fields?
+
+SGO imports all AWS tags automatically. Custom application-level tags are not currently supported but could be added in future versions.
+
+## Security
+
+### Is SGO secure?
+
+SGO is secure for **local use only**:
+- ✅ Read-only access to AWS
+- ✅ Uses standard AWS SDK authentication
+- ✅ Local data storage
+- ❌ **NO authentication/authorization**
+- ❌ **NOT safe for network exposure**
+
+### Should I use SGO on a shared server?
+
+**No.** Use SGO only on your personal workstation. Anyone with access to the running application can view all imported AWS data.
+
+### Can I add authentication?
+
+Not currently supported. SGO is designed for single-user local use. Adding authentication would significantly complicate the setup for the primary use case.
+
+### What about HTTPS?
+
+Not needed for local use. Since SGO runs on `localhost` and should never be exposed to networks, HTTPS is unnecessary.
+
+## Updates & Versioning
+
+### How do I update SGO?
+
+```bash
+# Pull latest changes
+git pull
+
+# Rebuild container
+docker-compose down
+docker-compose up --build
+```
+
+### Will updates break my data?
+
+Database schema is designed for forward compatibility. Major version updates will include migration instructions if needed.
+
+### How do I check my version?
+
+Check the repository for version tags or commits:
+```bash
+git log -1 --oneline
+```
+
+## License & Commercial Use
+
+### Can my company use SGO?
+
+Commercial use requires a paid license. Open an issue in the repository for licensing inquiries.
+
+### What counts as commercial use?
+
+Using SGO at a company, for business purposes, or as part of commercial operations requires a license. Personal/educational use is free.
+
+### Can I sell a modified version?
+
+No. The license explicitly prohibits modifying SGO for resale or commercial distribution.
+
+## Getting More Help
+
+### Where can I get support?
+
+1. Check this FAQ
+2. Read [Troubleshooting](Troubleshooting.md)
+3. Search existing issues in the repository
+4. Open a new issue with details
+
+### How do I report a bug?
+
+Open an issue in the repository with:
+- Description of the problem
+- Steps to reproduce
+- Expected vs actual behavior
+- Logs from `docker-compose logs`
+- Your environment (OS, Docker version)
+
+### Can I request features?
+
+Yes! Open a feature request issue. Include:
+- Use case description
+- Why the feature is needed
+- How you envision it working
+
+### Is there a community chat?
+
+Check the repository for links to community discussions or chat platforms.

wiki/Home.md

@@ -0,0 +1,96 @@
+# SGO: Security Groups Observatory
+
+A web-based tool for exploring AWS EC2 instances and Security Groups with direct AWS import, MFA support, and CSV export capabilities.
+
+## ⚠️ Security Warning
+
+**This application is designed for LOCAL USE ONLY. Do NOT expose it to the internet.**
+
+- SGO has no authentication or authorization mechanisms
+- It provides direct access to your AWS infrastructure data
+- It reads AWS credentials from your local system
+- Exposing it publicly would allow unauthorized access to sensitive AWS information
+
+**Always run on localhost (127.0.0.1) only. Never expose port 5000 to external networks.**
+
+## Features
+
+- **Direct AWS Import**: Import data directly from AWS using `~/.aws/config` with MFA/OTP support
+- **Parallel Import**: Import from multiple AWS accounts simultaneously
+- **Search & Filter**: Search by EC2 name, SG name, instance ID, group ID, or IP address
+- **Regex Search**: Enable regex checkbox for advanced pattern matching
+- **Filter by Type**: View all resources, only EC2 instances, or only Security Groups
+- **CSV Export**: Export search results, EC2 details, SG details, and security group rules to CSV
+- **Detailed Views**:
+  - **EC2 View**: Shows EC2 instance details with nested boxes for attached Security Groups
+  - **Security Group View**: Shows SG details with nested boxes for all attached EC2 instances
+- **Security Group Rules**: View and search ingress/egress rules for any security group
+- **Statistics Dashboard**: Quick overview of total SGs, EC2s, and accounts
+
+## Quick Links
+
+- [Quick Start Guide](Quick-Start.md)
+- [Configuration Options](Configuration.md)
+- [Usage Guide](Usage.md)
+- [AWS Configuration](AWS-Configuration.md)
+- [Troubleshooting](Troubleshooting.md)
+
+## Data Structure
+
+### Security Groups Table
+- Account ID & Name
+- Group ID & Name
+- Tag Name
+- Wave Tag
+- Git Repo Tag
+- Ingress Rule Count
+
+### EC2 Instances Table
+- Account ID & Name
+- Instance ID
+- Tag Name
+- State (running, stopped, etc.)
+- Private IP Address
+- Security Groups (IDs and Names)
+- Git Repo Tag
+
+## File Structure
+
+```
+sgo/
+├── app.py                      # Flask web application
+├── import_from_aws.py          # AWS direct import functions
+├── import_data.py              # CSV to SQLite import (legacy)
+├── requirements.txt            # Python dependencies
+├── Dockerfile                  # Container image definition
+├── docker-compose.yml          # Container orchestration (Docker volume)
+├── docker-compose.local.yml    # Alternative with local directory storage
+├── entrypoint.sh               # Container entrypoint with PUID/PGID support
+├── .dockerignore               # Files to exclude from container
+├── .env.example                # Example environment configuration
+├── .gitignore                  # Git ignore patterns
+├── README.md                   # This file
+├── data/                       # Local data directory (if using local mode)
+│   └── aws_export.db           # SQLite database
+├── static/
+│   ├── css/
+│   │   └── style.css           # Application styles
+│   └── images/
+│       └── logo.svg            # Application logo
+└── templates/
+    ├── import.html             # Import/profile selection page
+    └── index.html              # Main explorer interface
+```
+
+## License
+
+This project is dual-licensed:
+
+- **FREE** for individual, personal, educational, and non-commercial use
+- **PAID LICENSE REQUIRED** for commercial use by businesses and organizations
+
+You may NOT modify this software for the purpose of selling or commercially distributing it.
+
+See the [LICENSE](../LICENSE) file for full details.
+
+For commercial licensing inquiries, please open an issue in the repository.

wiki/Quick-Start.md

@@ -0,0 +1,137 @@
+# Quick Start Guide
+
+## TL;DR - Get Started in 30 Seconds
+
+```bash
+# 1. Create .env file with your AWS credentials path
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+EOF
+
+# 2. Start the container
+docker-compose up --build
+# or with Podman:
+podman-compose up --build
+
+# 3. Open browser to http://localhost:5000
+
+# 4. Select AWS profiles, enter MFA codes, and import!
+```
+
+## Container Setup (Recommended)
+
+The easiest way to run SGO is using Docker or Podman. Works on Linux, macOS, and Windows.
+
+### Prerequisites
+
+Install either:
+- **Docker**: https://docs.docker.com/get-docker/
+- **Podman**: https://podman.io/getting-started/installation
+
+### Setup Steps
+
+1. **Create environment configuration:**
+
+```bash
+# Copy the example file
+cp .env.example .env
+
+# Edit with your settings
+nano .env  # or your preferred editor
+```
+
+Or create it manually:
+
+```bash
+cat > .env << EOF
+AWS_CONFIG_PATH=$HOME/.aws
+PUID=$(id -u)
+PGID=$(id -g)
+EOF
+```
+
+2. **Start the application:**
+
+```bash
+# Docker
+docker-compose up --build
+
+# Podman
+podman-compose up --build
+```
+
+3. **Access the application:**
+
+Open your browser to `http://localhost:5000`
+
+### Import Data via GUI
+
+1. Open your browser to `http://localhost:5000`
+2. You'll see the **Import Page** with all your AWS profiles
+3. **Select profiles**: Check the AWS accounts you want to import
+4. **Enter MFA codes**: Paste your MFA/OTP codes for profiles that require authentication
+5. **Click "Start Import"**: Watch real-time progress as data is fetched **in parallel**
+6. **Auto-redirect**: When complete, you're taken to the Explorer
+
+**Parallel Import**: All selected profiles are imported simultaneously in separate threads, so total time is the max of any single import, not the sum. This prevents MFA timeout issues.
+
+### Explore Your Data
+
+- Search for EC2 instances and Security Groups
+- View detailed information
+- Inspect security group rules
+- Filter and search using regex
+- Export data to CSV
+
+### Refresh Data
+
+- Click the **Refresh Data** button to refresh data using cached AWS sessions (valid for 55 minutes)
+- Click the **Change Profiles** button to switch to different AWS accounts
+
+## Local Python Setup
+
+If you prefer to run without containers:
+
+### 1. Install Dependencies
+
+```bash
+python3 -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+pip install -r requirements.txt
+```
+
+### 2. Start the Application
+
+```bash
+python app.py
+```
+
+### 3. Open Browser
+
+Navigate to `http://localhost:5000`
+
+## Stopping the Application
+
+```bash
+# Stop with Ctrl+C, or:
+docker-compose down  # Docker
+podman-compose down  # Podman
+
+# To also remove the data volume:
+docker-compose down -v
+```
+
+## Important Notes
+
+- **⚠️ LOCAL USE ONLY**: Never expose this application to the internet. It has no authentication and provides access to sensitive AWS data.
+- **Database Persistence**: When using containers, the database persists in a Docker volume or `./data` directory
+- **Session Caching**: AWS sessions are cached for 55 minutes, allowing multiple refreshes without re-authentication
+- **Parallel Import**: All selected AWS accounts are imported simultaneously for maximum speed
+
+## Next Steps
+
+- [Configuration Options](Configuration.md) - Customize your setup
+- [AWS Configuration](AWS-Configuration.md) - Set up MFA and AWS profiles
+- [Usage Guide](Usage.md) - Learn how to use SGO features

wiki/README.md

@@ -0,0 +1,58 @@
+# SGO Wiki
+
+This directory contains comprehensive documentation for SGO (Security Groups Observatory).
+
+## Wiki Pages
+
+### Getting Started
+- **[Home](Home.md)** - Project overview, features, and file structure
+- **[Quick Start](Quick-Start.md)** - Get up and running in 30 seconds
+
+### Configuration
+- **[Configuration](Configuration.md)** - Environment variables, data storage options, and configuration examples
+- **[AWS Configuration](AWS-Configuration.md)** - AWS profile setup, MFA configuration, and IAM permissions
+
+### Usage
+- **[Usage Guide](Usage.md)** - Complete guide to using SGO features including search, views, and CSV exports
+- **[FAQ](FAQ.md)** - Frequently asked questions
+
+### Help
+- **[Troubleshooting](Troubleshooting.md)** - Common issues and solutions
+
+## Using These Docs
+
+### For New Users
+
+1. Start with [Quick Start](Quick-Start.md) to get SGO running
+2. Read [AWS Configuration](AWS-Configuration.md) to set up your AWS profiles
+3. Check [Usage Guide](Usage.md) to learn the features
+
+### For Troubleshooting
+
+1. Check [FAQ](FAQ.md) for common questions
+2. Review [Troubleshooting](Troubleshooting.md) for specific error messages
+3. If still stuck, open an issue with logs and details
+
+### For Configuration
+
+1. See [Configuration](Configuration.md) for all environment variables
+2. Review configuration examples for your use case
+3. Check platform-specific notes if on Windows/macOS/Linux
+
+## Contributing to Documentation
+
+Found an error or want to improve the docs?
+
+1. Edit the relevant markdown file
+2. Submit a pull request
+3. Follow the contribution guidelines in [CONTRIBUTING.md](../CONTRIBUTING.md)
+
+## Publishing to Wiki
+
+These markdown files can be:
+- Used directly in the repository
+- Published to GitHub/Codeberg wiki pages
+- Rendered with any markdown viewer
+- Hosted on documentation platforms
+
+For Codeberg, these files can be copied to the repository's wiki section.

wiki/Troubleshooting.md

@@ -0,0 +1,442 @@
+# Troubleshooting
+
+## Container Issues
+
+### Cannot Connect to Docker Daemon
+
+**Error**: `Cannot connect to the Docker daemon at unix:///var/run/docker.sock`
+
+**Solution**:
+```bash
+# Linux: Start Docker service
+sudo systemctl start docker
+sudo systemctl enable docker  # Start on boot
+
+# macOS/Windows: Open Docker Desktop application
+# Ensure Docker Desktop is running in system tray
+```
+
+### Port Already in Use
+
+**Error**: `Bind for 0.0.0.0:5000 failed: port is already allocated`
+
+**Solution**:
+
+Option 1 - Use different port:
+```bash
+# Add to .env file
+SGO_PORT=8080
+
+# Restart container
+docker-compose down && docker-compose up
+```
+
+Option 2 - Find and stop conflicting service:
+```bash
+# Linux/macOS: Find process using port 5000
+sudo lsof -i :5000
+sudo kill -9 <PID>
+
+# Windows: Find process using port 5000
+netstat -ano | findstr :5000
+taskkill /PID <PID> /F
+```
+
+### AWS Credentials Not Found in Container
+
+**Error**: `AWS config file not found` or `Permission denied: '/home/sgo/.aws/config'`
+
+**Solution**:
+
+1. Verify `.env` file exists and contains `AWS_CONFIG_PATH`:
+   ```bash
+   cat .env
+   # Should show: AWS_CONFIG_PATH=/home/yourusername/.aws
+   ```
+
+2. Create `.env` file if missing:
+   ```bash
+   cat > .env << EOF
+   AWS_CONFIG_PATH=$HOME/.aws
+   PUID=$(id -u)
+   PGID=$(id -g)
+   EOF
+   ```
+
+3. Verify AWS files exist on host:
+   ```bash
+   ls -la ~/.aws/
+   # Should show: config, credentials
+   ```
+
+4. Check file permissions:
+   ```bash
+   chmod 600 ~/.aws/credentials
+   chmod 644 ~/.aws/config
+   ```
+
+5. Rebuild container:
+   ```bash
+   docker-compose down && docker-compose up --build
+   ```
+
+### Permission Denied Errors
+
+**Error**: `PermissionError: [Errno 13] Permission denied: '/app/data/aws_export.db'`
+
+**Solution**:
+
+1. Set PUID/PGID to match your user:
+   ```bash
+   cat > .env << EOF
+   AWS_CONFIG_PATH=$HOME/.aws
+   PUID=$(id -u)
+   PGID=$(id -g)
+   EOF
+   ```
+
+2. If using local directory mode, fix ownership:
+   ```bash
+   sudo chown -R $(id -u):$(id -g) ./data
+   ```
+
+3. Or switch to Docker volume mode (default):
+   ```bash
+   # Use default docker-compose.yml
+   docker-compose down -v
+   docker-compose up --build
+   ```
+
+### Database Locked Error
+
+**Error**: `database is locked` or `OperationalError: database is locked`
+
+**Solution**:
+
+1. Stop all running containers:
+   ```bash
+   docker-compose down
+   ```
+
+2. If using local directory mode, remove lock file:
+   ```bash
+   rm -f ./data/aws_export.db-journal
+   ```
+
+3. If problem persists, use Docker volume mode:
+   ```bash
+   # Edit docker-compose.yml to use volume instead of bind mount
+   docker-compose up --build
+   ```
+
+### Container Keeps Restarting
+
+**Error**: Container exits immediately or restarts constantly
+
+**Solution**:
+
+1. Check container logs:
+   ```bash
+   docker-compose logs -f
+   ```
+
+2. Look for error messages in logs and address specific issues
+
+3. Try running in debug mode:
+   ```bash
+   # Add to .env
+   DEBUG=true
+
+   docker-compose down && docker-compose up
+   ```
+
+## AWS Configuration Issues
+
+### No AWS Profiles Found
+
+**Error**: "No AWS profiles found" or empty profile list
+
+**Solution**:
+
+1. Verify `~/.aws/config` exists:
+   ```bash
+   cat ~/.aws/config
+   ```
+
+2. Ensure profiles are properly formatted:
+   ```ini
+   [profile my-profile]
+   region = us-west-2
+
+   [default]
+   region = us-east-1
+   ```
+
+3. Check file permissions:
+   ```bash
+   chmod 644 ~/.aws/config
+   ```
+
+4. Restart container to reload profiles:
+   ```bash
+   docker-compose restart
+   ```
+
+### MFA Authentication Fails
+
+**Error**: "MFA authentication failed" or "Invalid MFA code"
+
+**Solution**:
+
+1. Verify MFA code is current:
+   - MFA codes expire every 30 seconds
+   - Generate a fresh code and try again
+
+2. Check `mfa_serial` is correct in `~/.aws/config`:
+   ```ini
+   [profile my-profile]
+   region = us-west-2
+   mfa_serial = arn:aws:iam::123456789012:mfa/username
+   ```
+
+3. Verify credentials are valid:
+   ```bash
+   aws sts get-caller-identity --profile my-profile
+   ```
+
+4. Ensure MFA device is active in AWS IAM Console
+
+5. Check for clock skew:
+   - Ensure system time is accurate
+   - MFA is time-based and requires synchronized clocks
+
+### Import Fails with Access Denied
+
+**Error**: "Access Denied" or "UnauthorizedOperation"
+
+**Solution**:
+
+1. Verify IAM permissions (see [AWS Configuration](AWS-Configuration.md#required-iam-permissions))
+
+2. Test permissions with AWS CLI:
+   ```bash
+   aws ec2 describe-instances --profile my-profile --max-results 1
+   aws ec2 describe-security-groups --profile my-profile --max-results 1
+   ```
+
+3. Check if credentials have expired:
+   - Temporary credentials expire
+   - Try refreshing credentials
+
+4. Verify you're using the correct profile
+
+### Wrong Region / No Resources Found
+
+**Error**: Import succeeds but no EC2/SG data appears
+
+**Solution**:
+
+1. Check region in profile configuration:
+   ```ini
+   [profile my-profile]
+   region = us-west-2  # Ensure this is correct
+   ```
+
+2. Remember: EC2 resources are region-specific
+
+3. Try specifying region explicitly:
+   ```bash
+   aws ec2 describe-instances --profile my-profile --region us-west-2
+   ```
+
+4. Import from correct region in profile
+
+## Application Issues
+
+### Search Returns No Results
+
+**Problem**: Search box doesn't show any results
+
+**Solution**:
+
+1. Verify data was imported successfully:
+   - Check import progress logs
+   - Look for "Import complete" messages
+
+2. Try different search terms:
+   - Minimum 2 characters required
+   - Try searching for "*" to see all resources
+
+3. Check filter buttons:
+   - Ensure you're not filtering out the type you're searching for
+   - Try "All Resources" button
+
+4. Refresh the page:
+   ```
+   Ctrl+F5 or Cmd+Shift+R
+   ```
+
+### Blank Page or UI Not Loading
+
+**Problem**: Browser shows blank page or incomplete UI
+
+**Solution**:
+
+1. Check browser console for JavaScript errors:
+   - Press F12 to open developer tools
+   - Look at Console tab for errors
+
+2. Clear browser cache and reload:
+   ```
+   Ctrl+Shift+R (Windows/Linux)
+   Cmd+Shift+R (macOS)
+   ```
+
+3. Try a different browser
+
+4. Check if application is running:
+   ```bash
+   docker-compose logs
+   ```
+
+### CSV Export Not Working
+
+**Problem**: Export button doesn't download file
+
+**Solution**:
+
+1. Check browser popup blocker settings
+
+2. Look for downloads in browser's download location
+
+3. Try different browser
+
+4. Check browser console for JavaScript errors (F12)
+
+### Session Expired / MFA Required Again
+
+**Problem**: Asked for MFA again before 55 minutes
+
+**Solution**:
+
+- AWS sessions expire after 1 hour (we cache for 55 minutes)
+- This is expected behavior for security
+- Simply enter MFA code again to create new session
+
+## Performance Issues
+
+### Slow Import
+
+**Problem**: Import takes a very long time
+
+**Solution**:
+
+1. **Use parallel imports**:
+   - Select all profiles at once
+   - Click "Start Import" on each profile
+   - All imports run simultaneously
+
+2. **Check network connectivity to AWS**:
+   ```bash
+   ping ec2.us-west-2.amazonaws.com
+   ```
+
+3. **Large accounts take longer**:
+   - Many EC2 instances/security groups increase import time
+   - This is normal
+
+### Slow Search
+
+**Problem**: Search is laggy or slow
+
+**Solution**:
+
+1. **Large datasets**:
+   - Thousands of resources can slow search
+   - This is a known limitation of client-side search
+
+2. **Use more specific search terms**:
+   - Narrow down results with longer search strings
+
+3. **Use regex carefully**:
+   - Complex regex patterns can be slow
+   - Disable regex for simple searches
+
+## Platform-Specific Issues
+
+### Windows
+
+**Problem**: Path issues or permission errors
+
+**Solution**:
+
+1. Use WSL2 for better compatibility:
+   ```powershell
+   # Install WSL2 and run commands from Linux environment
+   wsl --install
+   ```
+
+2. Use Git Bash for running commands
+
+3. Ensure Docker Desktop is using WSL2 backend
+
+4. Use forward slashes in paths
+
+### macOS
+
+**Problem**: Docker can't access files in home directory
+
+**Solution**:
+
+1. Open Docker Desktop Settings
+2. Go to Resources → File Sharing
+3. Add your home directory to shared paths
+4. Click "Apply & Restart"
+
+### Linux (Podman)
+
+**Problem**: `podman-compose` not found or issues
+
+**Solution**:
+
+1. Install podman-compose:
+   ```bash
+   # Fedora
+   sudo dnf install podman-compose
+
+   # Ubuntu/Debian
+   pip3 install podman-compose
+   ```
+
+2. Use `docker` command instead:
+   ```bash
+   # Podman supports docker CLI
+   alias docker=podman
+   docker-compose up --build
+   ```
+
+3. Check SELinux contexts (Fedora/RHEL):
+   ```bash
+   # Already handled by :z flag in docker-compose.yml
+   # If issues persist, try:
+   sudo setenforce 0  # Temporarily permissive mode for testing
+   ```
+
+## Getting Help
+
+If you continue to experience issues:
+
+1. **Check application logs**:
+   ```bash
+   docker-compose logs -f
+   ```
+
+2. **Enable debug mode** (see [Configuration](Configuration.md#example-4-debug-mode))
+
+3. **Open an issue** on the repository with:
+   - Error messages from logs
+   - Steps to reproduce
+   - Your environment (OS, Docker/Podman version)
+   - Relevant configuration (sanitized)
+
+4. **Check existing issues** - Your problem may already be solved

wiki/Usage.md

@@ -0,0 +1,140 @@
+# Usage Guide
+
+## Search
+
+### Basic Search
+
+1. Type in the search box (minimum 2 characters)
+2. Results appear instantly as you type
+3. Filter by resource type using the buttons: **All Resources** | **EC2 Instances** | **Security Groups**
+
+### Regex Search
+
+Enable advanced pattern matching:
+
+1. Check the "Regex" box to use regular expressions
+2. Examples:
+   - `^prod-.*-\d+$` - Finds names starting with "prod-" and ending with numbers
+   - `(dev|test|qa)` - Finds names containing dev, test, or qa
+   - `10\.0\.\d+\.\d+` - Finds IP addresses in the 10.0.x.x range
+
+## View Details
+
+### EC2 Instance View
+
+Click on any EC2 instance from search results to view:
+
+- **Main card** shows EC2 details:
+  - Instance ID
+  - Private IP Address
+  - State (running, stopped, etc.)
+  - Account ID and Name
+  - All AWS Tags
+- **Nested cards** show all attached Security Groups with their details
+
+### Security Group View
+
+Click on any Security Group from search results to view:
+
+- **Main card** shows SG details:
+  - Group ID
+  - Group Name
+  - Wave Tag
+  - Ingress/Egress Rule Counts
+  - All AWS Tags
+- **Nested cards** show all EC2 instances using this Security Group
+
+## View Security Group Rules
+
+When viewing security groups (either attached to an EC2 or directly):
+
+1. Click the **View Rules** button on any security group card
+2. A modal opens showing all ingress and egress rules
+3. Switch between **Ingress** and **Egress** tabs
+4. Use the search box to filter rules by protocol, port, source, or description
+5. Rules are displayed in a compact table format with:
+   - Protocol (TCP, UDP, ICMP, All)
+   - Port Range
+   - Source Type (CIDR, Security Group, Prefix List)
+   - Source (IP range or SG ID)
+   - Description
+
+## Navigation
+
+- Click **← Back to Search** to return to search results
+- Perform a new search at any time
+- Click outside the rules modal to close it
+
+## Export to CSV
+
+SGO provides comprehensive CSV export capabilities in multiple contexts.
+
+### Search Results Export
+
+- Click the **💾 Export** button in the view controls (top right)
+- Exports all current search results with filters applied
+- Includes:
+  - Type (EC2 or Security Group)
+  - Name
+  - ID (Instance ID or Group ID)
+  - Account
+  - State (for EC2)
+  - IP Address (for EC2)
+  - Security Groups count (for EC2)
+  - Wave
+  - Git Repository
+  - Git Organization
+  - Git File
+
+### EC2 Instance Details Export
+
+- Click the **💾 Export** button in any EC2 detail card
+- Exports complete EC2 information including:
+  - Instance details (ID, name, state, IP, account info)
+  - All AWS tags
+  - Attached security groups with their details
+
+### Security Group Details Export
+
+- Click the **💾 Export** button in any SG detail card
+- Exports complete SG information including:
+  - Group details (ID, name, wave, rule counts)
+  - All AWS tags
+  - Attached EC2 instances with their details
+
+### Security Group Rules Export
+
+- Click the **💾 Export** button in the rules modal
+- Exports all ingress and egress rules with:
+  - Rule details (direction, protocol, ports, source)
+  - Group ID and name
+  - Account ID and name
+  - Git file and commit information from tags
+
+All exports include timestamps in filenames and proper CSV escaping for safe data handling.
+
+## Import and Refresh
+
+### Import from AWS
+
+1. From the Explorer page, click **Change Profiles** (or visit `/` directly)
+2. Select AWS profiles to import
+3. Enter MFA/TOTP codes for profiles that require authentication
+4. Click **Start Import** for each profile
+5. Watch real-time progress logs
+6. Click **Done - Go to Explorer** when finished
+
+### Refresh Data
+
+- Click the **Refresh Data** button on the Explorer page
+- Uses cached AWS sessions (valid for 55 minutes)
+- No need to re-enter MFA codes during the session cache window
+- Imports data from the same profiles used in the last import
+
+## Tips
+
+- **Parallel Imports**: Import multiple profiles simultaneously to save time
+- **Session Cache**: Refresh multiple times within 55 minutes without re-entering MFA
+- **Regex Power**: Use regex patterns for complex searches across your infrastructure
+- **CSV Exports**: Export at any level (search results, individual resources, or rules)
+- **Quick Navigation**: Use browser back/forward buttons or the built-in navigation links

wiki/_Sidebar.md

@@ -0,0 +1,20 @@
+# SGO Wiki
+
+## Getting Started
+- [Home](Home.md)
+- [Quick Start](Quick-Start.md)
+
+## Configuration
+- [Configuration Options](Configuration.md)
+- [AWS Configuration](AWS-Configuration.md)
+
+## Using SGO
+- [Usage Guide](Usage.md)
+- [FAQ](FAQ.md)
+
+## Help
+- [Troubleshooting](Troubleshooting.md)
+
+## Resources
+- [Main Repository](https://codeberg.org/edfig/SGO)
+- [Report Issue](https://codeberg.org/edfig/SGO/issues)