Inside Our $10K Paper Trading Bot: Design and Lessons

We said we'd build in public. Not the curated kind — the real kind. So here it is: a look at how our trading bot is designed and what we learned building it.

Simulated paper trading, not live results; not financial advice. Earlier versions of this report quoted a trade count and other figures that we could not tie to a retained run record, so they have been removed. We have since reconstructed the results from the ledger and logs we kept: the simulated run (Feb 26 to Apr 8, 2026) ended net negative. See the retired-project case study for the figures and caveats. No real money was traded.

What the Bot Actually Did

The bot is built to trade simulated positions across two markets: cryptocurrency spot trading (BTC/USDT, ETH/USDT, SOL/USDT) and Polymarket prediction contracts. Every single decision went through the same pipeline:

1. Market Data Collection — Price, volume, order book depth, funding rates, open interest, derivatives data (put/call ratios, implied volatility), and on-chain metrics for crypto. Polymarket odds, volume, and resolution timelines for prediction markets.
2. Strategy Scoring — Nine strategies evaluated in parallel: mean reversion, RSI swing trading, MACD crossovers, Bollinger Bounce, volume breakouts, DCA laddering, minute scalping, agentic LLM analysis, and Polymarket market making.
3. LLM Reasoning — Claude Opus (primary) or Qwen3.5-plus (fallback) analyzes the regime, evaluates confluence between strategies, and produces a decision with confidence score.
4. Critic Review — A separate critic model checks for contradictions, excessive risk, consecutive losses, and drawdown thresholds. Hard rules can veto trades automatically.
5. Execution or Queue — High-confidence actions (85%+) execute immediately. Lower-confidence actions queue for human review via Telegram.

The result: 24 close actions, 6 buy orders, 1 sell order, and 1,550+ hold decisions where the bot correctly chose to do nothing.

What Worked

1. Mean Reversion in Ranging Markets

When the market chops sideways, mean reversion tends to fit. The bot identifies ranging regimes using ADX, Bollinger Band width, and price distance from EMA50, and in those conditions fades extremes.

The key insight: regime matters more than strategy. A strategy that works beautifully in a ranging market gets crushed in a breakout. We built regime awareness into the core decision loop.

2. An LLM Critic as a Risk Check

The critic model isn't just a checkbox. It is designed to veto bad trades:

• Consecutive loss veto — After a set number of consecutive losses, a hard rule vetoes further buys until human review.
• Drawdown circuit breaker — Past a max-drawdown threshold, the critic vetoes new positions.
• Regime mismatch detection — When the LLM suggested buys in a confirmed downtrend (price below EMA50, ADX > 25, DI- > DI+), the probabilistic veto flags it.

Most critic outputs are "monitor" recommendations: the design favors watching over acting.

3. Confidence Thresholds Work

We set auto-execute at 85% confidence. Everything else queues for human review. This is intended to limit drawdowns:

• Trades with confidence < 0.65 were automatically downgraded to "hold" or "queue".
• When confidence dropped below 0.45, the fallback mechanism kicked in — defaulting to hold rather than forcing a low-conviction trade.

What Broke (and What We Learned)

1. LLM Hallucination Risk

Early in the test, we saw the LLM make confident but wrong calls. Example: suggesting buys when derivatives data showed bearish hedging flow (put/call ratio > 0.85). The fix:

• Sanitized prompts — We strip out any reasoning that could be manipulated by injected content.
• Contradiction checks — The LLM must explicitly confirm its reasoning doesn't conflict with the data.
• Critic disagreement escalation — When the critic disagrees twice in a row, the trade escalates to human review.

2. Provider Outages

Provider outages happen. When Claude Opus timed out, the fallback to Qwen3.5-plus via DashScope kicked in. When both failed, the bot defaulted to hold.

Lesson: never depend on a single LLM provider. We now have three tiers: Claude Opus (primary), Qwen3.5-plus (secondary), and local Ollama models (emergency fallback).

3. Overtrading in First Week

The bot initially wanted to trade too frequently. We added:

• Cluster detection — If 3+ actions occur within 15 minutes, the bot pauses and requires review.
• Same-symbol cooldown — a minimum wait between trades on the same pair.
• Daily budget cap — $1.50 maximum in simulated trading fees per day.

The Tech Stack (What Actually Runs It)

Here's the stack the bot was built on:

• Python + asyncio — 15-minute polling loop (900 seconds), with fast mode at 300 seconds during high volatility.
• LLM Stack — Claude Opus via API (primary), Qwen3.5-plus via DashScope (secondary), Qwen2:7b and Gemma3:4b via Ollama (local fallback).
• Data Sources — 12+ APIs: crypto exchanges, Polymarket, Deribit for options flow, Fear & Greed Index, LI.FI for cross-chain arbitrage scans.
• Infrastructure — Runs on WSL (Windows Subsystem for Linux) on a laptop. No AWS. No Kubernetes. Systemd user service with automatic restart on crash.
• Monitoring — Telegram bot for alerts, Next.js dashboard at hub.tacavar.com for real-time visibility.

Total infrastructure cost: $0/month (existing hardware). The bot runs on the same machine Josh uses for development.

Why Paper Trading Matters

Every trade in this report is simulated. The bot operates with dry_run = true — a flag hardcoded at multiple levels:

• Config file: dry_run is true
• Execution layer: refuses to send real orders if dry_run is false
• Prompt sanitization: removes any instructions that could override the paper trading constraint

We believe in earning the right to trade with real capital. Before we flip the switch, we need to see:

• Consistent profitability over 6-12 months
• Max drawdown < 15%
• Win rate > 45% with positive risk/reward
• No catastrophic failures (circuit breakers must hold)

We're not there yet. But we're learning. And we're not risking anyone's capital while we figure it out.

What's Next

The roadmap for the next 30 days:

• Auto-tuning — Strategies that underperform get automatically deprioritized. We're building a decay mechanism that reduces position sizing for weak strategies.
• Polymarket Market Making — We've enabled MM logic with spread targeting, inventory management, and toxic flow detection. Testing in paper mode first.
• Copycat Strategies — Tracking top Polymarket traders and mimicking their positions (with LLM validation).
• Better Visualizations — The dashboard needs equity curves, win/loss heatmaps, and strategy attribution.

The Point of Building in Public

Transparency builds trust. In an industry full of black boxes and unverifiable claims, we're showing exactly how our system works.

We're also holding ourselves accountable. When you build in public, you can't hide failures. Every loss is logged. Every decision is recorded. This discipline makes us better traders.

And honestly? We hope this inspires others. The trading bot space doesn't need more hype. It needs more transparency, more open source collaboration, and more honest conversations about what works and what doesn't.

We'll keep sharing updates. We'll keep sharing what we can verify, wins and losses alike.

That's the point. Building in the open.