Deploy IOanyT SecureAI

This guide takes you from subscribing on AWS Marketplace to a running, secured AI API, and covers day-2 operations. Everything in SecureAI auto-configures on first boot — there is no manual setup — so most of the steps below are verifying the stack and retrieving your credentials.

Prerequisites

• An AWS account with permission to subscribe to AWS Marketplace products and launch EC2 instances.
• A GPU-capable instance quota in your target region (SecureAI is validated on the g4dn family / NVIDIA T4). Request a quota increase for “Running On-Demand G and VT instances” if needed.
• An EC2 key pair for SSH access.
• A tool to make HTTPS requests (curl, or any OpenAI-compatible client).

Step 1 — Subscribe & launch

Console

1. Open the SecureAI listing on AWS Marketplace and choose Continue to Subscribe → Continue to Configuration → Continue to Launch.
2. Select a recommended instance type — g4dn.xlarge or g4dn.2xlarge.
3. Choose your key pair.
4. Configure the security group (see Step 2) — open ports 443 and 22.
5. Launch.

CLI

aws ec2 run-instances \
  --image-id <SECUREAI_AMI_ID> \
  --instance-type g4dn.xlarge \
  --key-name <YOUR_KEY_PAIR> \
  --security-group-ids <YOUR_SG_ID> \
  --region us-east-1

Replace <SECUREAI_AMI_ID> with the AMI ID shown on the Marketplace listing’s launch page for your region.

Step 2 — Security group / ports

Open only what you need. SecureAI uses two ports:

Port	Protocol	Purpose	Recommended source
443	TCP	HTTPS — LLM API, Qdrant, all app traffic (Caddy, TLS)	Your app/clients’ IP ranges
22	TCP	SSH — key-only admin access	Your admin IP only

Do not open port 22 to 0.0.0.0/0. Restrict SSH to your own IP, and restrict 443 to the clients that will call the API.

Step 3 — Connect via SSH

ssh -i "your-key.pem" ubuntu@<PUBLIC_IP>

If you get a permission error on the key file:

chmod 400 your-key.pem

First launch: allow up to 10 minutes for the AI stack to initialize and the model to load (a one-time EBS warm-up). Later restarts are fast.

Step 4 — Retrieve your per-instance API keys

Each instance generates its own unique keys on first boot:

sudo cat /opt/ioanyt/.env

This shows:

• VLLM_API_KEY — for the LLM inference API
• QDRANT_API_KEY — for the vector database

Store these securely (e.g. AWS Secrets Manager) and never commit them.

Step 5 — Verify LLM inference (OpenAI-compatible)

curl -k https://<PUBLIC_IP>/v1/chat/completions \
  -H "Authorization: Bearer <VLLM_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen2.5-3B-Instruct","messages":[{"role":"user","content":"Hello"}]}'

A JSON completion response confirms the inference stack is live. Because the endpoint is OpenAI-compatible, you can point any OpenAI SDK at https://<PUBLIC_IP>/v1 using VLLM_API_KEY as the API key.

Step 6 — Verify the RAG vector database (Qdrant)

curl -k https://<PUBLIC_IP>/qdrant/collections \
  -H "api-key: <QDRANT_API_KEY>"

Day-2 operations

Manage the stack

# Check container status
sudo docker compose -f /opt/ioanyt/docker-compose.yml ps

# Restart the full GenAI stack
sudo systemctl restart ioanyt-genai

Backups

• Snapshot the EBS volume on your own schedule for point-in-time recovery.
• Back up any Qdrant collections you populate (they persist on the instance’s volume).

Monitoring

• auditd is enabled for system audit logging.
• Forward instance logs/metrics to CloudWatch (or your SIEM) as needed.

Patching

SecureAI ships hardened, but ongoing OS and dependency patching after launch is your responsibility under the AWS shared-responsibility model. Apply security updates on your normal cadence and re-validate the stack afterward.

Upgrading to a new AMI version

New SecureAI versions are published as new AMI versions on the Marketplace listing. To upgrade, launch a fresh instance from the new version, migrate your Qdrant data and configuration, then decommission the old instance. See the changelog for version history.

Troubleshooting

Symptom	Likely cause	Fix
API not responding right after launch	Stack still initializing / model loading	Wait up to 10 minutes on first boot, then retry Step 5
Permission denied (publickey) on SSH	Key file permissions too open	chmod 400 your-key.pem
Connection refused on 443	Security group not open, or stack restarting	Confirm SG allows 443 from your IP; check docker compose ps
401 / auth error on API call	Wrong or missing key	Re-read keys with sudo cat /opt/ioanyt/.env
GPU not detected	Non-GPU instance type	Relaunch on a g4dn instance

Support

Email support@ioanyt.com for help.