How I Built My Dual-Model AI Security & OSINT Agent

Project Goal

I built this project to create a local AI-powered security research assistant that can help with authorized bug bounty testing, OSINT workflows, and vulnerability verification.

The goal was not just to build a chatbot. I wanted an agent that could:

• Understand security tasks from natural language
• Use controlled Python tools
• Enforce target scope and safety rules
• Combine GPT-5.5 and Claude Opus
• Run OSINT tools like Sherlock and Maigret
• Verify vulnerability hypotheses with evidence
• Reduce false positives before reporting

1. Create the Main Project Directory

I started by creating a main project folder called:

mkdir ~/ai-agent
cd ~/ai-agent

This directory became the root of the entire agent.

The final structure looks like this:

ai-agent/
├── app.py
├── agent.py
├── config.py
├── schemas.py
├── policies.py
├── llm_clients.py
├── requirements.txt
├── .env
├── tools/
├── tests/
├── docs/
└── examples/

2. Create a Python Virtual Environment

I used a Python virtual environment so the project dependencies stay isolated.

cd ~/ai-agent
python3 -m venv venv
source venv/bin/activate

Then I installed dependencies:

pip install fastapi uvicorn pydantic python-dotenv openai anthropic httpx requests beautifulsoup4 PyYAML

3. Create .env

The .env file stores API keys, model names, and safety settings.

nano .env

Example:

OPENAI_API_KEY=your_openai_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
OPENAI_MODEL=gpt-5.5
ANTHROPIC_MODEL=claude-opus-4-8
DEFAULT_LLM_PROVIDER=openai
ENABLE_DUAL_LLM=true
ENABLE_RECON=true
ENABLE_ACTIVE_SCANNING=false
PENTEST_ALLOWLIST=nerminzlatanovic.com,www.nerminzlatanovic.com,localhost,127.0.0.1,testphp.vulnweb.com
MAX_CRAWL_DEPTH=2
NUCLEI_SEVERITY=low,medium,high,critical
NUCLEI_RATE_LIMIT=5

Purpose of .env: stores configuration without hardcoding secrets in Python files.

Important: never upload real .env files to GitHub.

4. Create config.py

config.py loads environment variables and makes them available to the rest of the project.

Purpose:

Central configuration file for API keys, model names, safety flags, and tool settings.

Example responsibilities:

• OPENAI_API_KEY
• ANTHROPIC_API_KEY
• OPENAI_MODEL
• ANTHROPIC_MODEL
• ENABLE_RECON
• ENABLE_ACTIVE_SCANNING
• PENTEST_ALLOWLIST

This file is important because every tool and workflow can read the same configuration.

5. Create schemas.py

schemas.py contains Pydantic models for tool inputs and outputs.

Purpose:

Defines structured input/output formats for every tool.

Example:

class AccessControlVerifierIn(BaseModel):
    url: str
    method: str = "GET"
    account_a_headers: Dict[str, str] = {}
    account_b_headers: Dict[str, str] = {}

Why this matters:

Every tool receives predictable input and returns predictable output. This makes the agent easier to debug, test, and extend.

6. Create policies.py

policies.py controls which tools are allowed and when they can run.

Purpose:

Safety and scope enforcement.

It defines things like:

• SAFE_TOOLS
• RECON_TOOLS
• ACTIVE_TOOLS

It also helps prevent the agent from running dangerous actions against targets that are not authorized.

Example safety behavior:

• target.com → blocked if not in allowlist
• nerminzlatanovic.com → allowed if listed in scope

7. Create llm_clients.py

llm_clients.py connects the project to OpenAI and Anthropic.

Purpose:

Provides reusable functions for GPT-5.5, Claude Opus, and dual-model calls.

Functions:

• call_openai()
• call_anthropic()
• call_llm()
• call_dual_llm()

How I use them:

• GPT-5.5 = main planner and tool router
• Claude Opus = second planner, reviewer, and false-positive challenger
• Dual mode = both models work together on selected tasks

8. Create app.py

app.py is the FastAPI web server.

Purpose:

Expose the agent through a local API endpoint.

Main route:

POST /chat

Example API flow:

Terminal command
↓
FastAPI /chat
↓
agent.py
↓
tool execution
↓
final response

I start the server with:

uvicorn app:app --reload

9. Create agent.py

agent.py is the brain of the project.

Purpose:

Routes user requests to the correct tools.

Handles GPT/Claude workflows.

Formats tool results.

Controls dual-agent mode.

It contains:

• Tool registry
• Tool handlers
• Tool manifest
• Keyword routing
• Dual-agent logic
• Output formatting
• Safety checks
• Claude review routing

Example commands handled by agent.py:

ai "verify cors on https://example.com"

ai "username enum @exampleuser"

ai "dual agent: validate headers on https://example.com"

ai "request diff: GET / HTTP/1.1
Host: example.com"

10. Create the tools/ Directory

The tools/ directory contains all custom security and OSINT tools.

mkdir tools

Each file in tools/ is a separate capability.

Example:

tools/access_control_verifier.py
tools/request_diff_engine.py
tools/username_enum.py

11. Create the tests/ Directory

The tests/ directory stores unit tests.

mkdir tests

Purpose:

Make sure tools work after every change.

Run tests:

python3 -m unittest discover -s tests

12. Create the ai Shell Helper

To talk to the agent from terminal, I created an ai() shell function.

I added this to ~/.zshrc:

ai() {
  jq -n --arg message "$*" '{message: $message}' |
  curl -s http://127.0.0.1:8000/chat     -H "Content-Type: application/json"     -d @- |
  jq -r '.result.final'
}

ai-debug() {
  jq -n --arg message "$*" '{message: $message}' |
  curl -s http://127.0.0.1:8000/chat     -H "Content-Type: application/json"     -d @-
}

Then reload:

source ~/.zshrc

Purpose:

• ai = clean final output
• ai-debug = full raw JSON response for debugging

13. Dual-Model Workflow

I implemented three modes.

GPT-5.5 Only

Normal commands use GPT-5.5 as the primary agent.

ai "verify cors on https://nerminzlatanovic.com"

Claude Only

Claude can review output directly.

ai "claude review: Username enumeration found several possible profiles. Review confidence."

Dual-Agent Mode

Both GPT-5.5 and Claude Opus plan together.

ai "dual agent: validate headers on https://nerminzlatanovic.com"

Dual-agent flow:

GPT-5.5 Plan
Claude Opus Plan
Tool Mapping
Tool Executed
Raw Evidence Summary
Claude Critique
Final Recommendation

14. Tool Inventory and Purpose

Core Files

Reconnaissance Tools

JavaScript and API Analysis Tools

Scanner and External Tool Wrappers

Candidate Mapping Tools

Vulnerability Verification Tools

These are the tools that move the project beyond recon.

{
  "isAdmin": true,
  "role": "admin",
  "verified": true,
  "plan": "enterprise"
}

OSINT / Trace Labs Tools

Reporting and Triage Tools

Utility Tools

How Someone Can Recreate This Project

Step 1: Create Directory

mkdir ~/ai-agent
cd ~/ai-agent

Step 2: Create Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3: Install Dependencies

pip install fastapi uvicorn pydantic python-dotenv openai anthropic httpx requests beautifulsoup4 PyYAML

Step 4: Create Core Files

touch app.py
touch agent.py
touch config.py
touch schemas.py
touch policies.py
touch llm_clients.py
touch requirements.txt
touch .env

Example: How One Python Tool Is Written

Below is a simplified example of how I wrote tools inside the agent. This example shows the structure of a safe verifier tool.

from pydantic import BaseModel, Field
from typing import Dict, List, Optional
import httpx


class HeaderVerifierIn(BaseModel):
    url: str = Field(..., description="Target URL to check")
    method: str = "HEAD"


class HeaderFinding(BaseModel):
    header: str
    status: str
    value: Optional[str] = None


class HeaderVerifierOut(BaseModel):
    tool: str
    target: str
    status_code: int
    verdict: str
    findings: List[HeaderFinding]
    recommendation: str


async def header_verifier(inp: HeaderVerifierIn) -> HeaderVerifierOut:
    async with httpx.AsyncClient(timeout=10, follow_redirects=True) as client:
        response = await client.request(
            inp.method,
            inp.url,
        )

    headers = {
        key.lower(): value
        for key, value in response.headers.items()
    }

    required_headers = [
        "strict-transport-security",
        "content-security-policy",
        "x-content-type-options",
        "x-frame-options",
        "referrer-policy",
        "permissions-policy",
    ]

    findings = []

    for header in required_headers:
        if header in headers:
            findings.append(
                HeaderFinding(
                    header=header,
                    status="present",
                    value=headers[header],
                )
            )
        else:
            findings.append(
                HeaderFinding(
                    header=header,
                    status="missing",
                    value=None,
                )
            )

    missing = [
        finding.header
        for finding in findings
        if finding.status == "missing"
    ]

    if missing:
        verdict = "Manual Verification Needed"
        recommendation = (
            "Review whether missing defensive headers are intentional. "
            "Add appropriate headers where needed."
        )
    else:
        verdict = "False Positive"
        recommendation = "All checked defensive headers were present."

    return HeaderVerifierOut(
        tool="header_verifier",
        target=inp.url,
        status_code=response.status_code,
        verdict=verdict,
        findings=findings,
        recommendation=recommendation,
    )

Explanation of the Code

The file starts by importing the Python libraries needed by the tool.

from pydantic import BaseModel, Field
from typing import Dict, List, Optional
import httpx

pydantic is used to define structured input and output models. httpx is used to send HTTP requests.

The input model defines what the tool accepts:

class HeaderVerifierIn(BaseModel):
    url: str
    method: str = "HEAD"

This means the tool needs a URL and can optionally accept an HTTP method. By default, it uses HEAD because header checks do not need to download the full page body.

The output models define what the tool returns:

class HeaderFinding(BaseModel):
    header: str
    status: str
    value: Optional[str] = None

Each header result includes the header name, whether it was present or missing, and its value if found.

The main tool function is asynchronous:

async def header_verifier(inp: HeaderVerifierIn) -> HeaderVerifierOut:

This allows the FastAPI agent to run the tool efficiently.

The tool sends the request:

response = await client.request(
    inp.method,
    inp.url,
)

Then it normalizes headers to lowercase so checks are consistent:

headers = {
    key.lower(): value
    for key, value in response.headers.items()
}

The tool checks for important defensive headers:

required_headers = [
    "strict-transport-security",
    "content-security-policy",
    "x-content-type-options",
    "x-frame-options",
    "referrer-policy",
    "permissions-policy",
]

For each header, it records whether the header exists:

if header in headers:
    status="present"
else:
    status="missing"

Finally, the tool decides a verdict:

if missing:
    verdict = "Manual Verification Needed"
else:
    verdict = "False Positive"

I intentionally used Manual Verification Needed instead of automatically claiming a vulnerability. Missing headers can be hardening issues, but they are not always reportable vulnerabilities by themselves.

Why I Use This Structure

Every tool follows the same basic pattern:

Input schema
↓
Safety-aware execution
↓
Evidence collection
↓
Verdict
↓
Recommendation
↓
Structured output

This makes the agent easier to expand. When I add a new tool, I create:

• Tool input model
• Tool output model
• Tool function
• Tool registration in agent.py
• Policy entry in policies.py
• Tests

This is how the project stays organized and safe while still allowing the agent to become more powerful.

Step 5: Create Folders

mkdir tools
mkdir tests
mkdir docs
mkdir examples

Step 6: Add API Keys

Create .env:

OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
OPENAI_MODEL=gpt-5.5
ANTHROPIC_MODEL=claude-opus-4-8
ENABLE_DUAL_LLM=true
ENABLE_RECON=true
ENABLE_ACTIVE_SCANNING=false
PENTEST_ALLOWLIST=localhost,127.0.0.1,yourdomain.com

Step 7: Build Tool Registry

In agent.py, register every tool:

HANDLERS = {
    "cors_verifier": (CorsVerifierIn, cors_verifier),
    "open_redirect_verifier": (OpenRedirectVerifierIn, open_redirect_verifier),
    "request_diff_engine": (RequestDiffEngineIn, request_diff_engine),
}

Step 8: Add Safety Rules

In policies.py, define allowed tools:

SAFE_TOOLS = {
    "username_pivot",
    "case_intake_parser",
    "jwt_session_analyzer",
}

RECON_TOOLS = {
    "http_probe",
    "security_headers",
    "cors_verifier",
    "open_redirect_verifier",
}

Step 9: Start FastAPI

uvicorn app:app --reload

Step 10: Add Shell Helper

Add this to ~/.zshrc:

ai() {
  jq -n --arg message "$*" '{message: $message}' |
  curl -s http://127.0.0.1:8000/chat     -H "Content-Type: application/json"     -d @- |
  jq -r '.result.final'
}

Reload:

source ~/.zshrc

Step 11: Test

ai "verify cors on https://yourdomain.com"

ai "dual agent: validate headers on https://yourdomain.com"

ai "username pivot for @exampleuser"

What Makes This Project Different

This project is different because it combines:

AI reasoning
+
real tools
+
scope enforcement
+
dual-model review
+
evidence-based verification

Instead of saying:

This might be vulnerable.

The goal is to say:

Here is the request. Here is the response. Here is the comparison. Here is the evidence. Here is the confidence. Here is what still needs manual verification.

That is what makes the project useful for real security work.

Final Disclaimer