Advanced Social Media Analysis Agent

A production-grade AI agent built on cutting-edge research patterns for intelligent social media analysis on the Apify platform.

Architecture Overview

This agent implements state-of-the-art AI agent patterns based on research papers and production best practices:

Core Patterns Implemented

1. ReAct (Reasoning + Acting) - Paper

   • Interleaves chain-of-thought reasoning with tool-using actions
   • Thought -> Action -> Observation cycle
   • 34% improvement over imitation learning on ALFWorld

2. Reflexion (Self-Improvement) - Paper

   • Verbal reinforcement learning through self-reflection
   • Episodic memory for storing lessons learned
   • 91% pass@1 on HumanEval (vs GPT-4's 80%)

3. 12-Factor Agents - GitHub

   • Production principles from 100+ SaaS builders
   • Small, focused agents (3-20 steps max)
   • Own your control flow
   • Compact errors into context for self-healing

4. Multi-Tier Memory System

   • Working Memory: Current conversation context
   • Episodic Memory: Task-specific learnings
   • Semantic Memory: General knowledge
   • Procedural Memory: How to perform tasks

5. Human-in-the-Loop

   • Approval gates for high-risk actions
   • Interrupt/resume capabilities
   • Confidence-based escalation

Features

• Intelligent Reasoning: ReAct pattern for transparent decision-making
• Self-Improvement: Reflexion pattern learns from each execution
• Advanced Memory: Multi-tier memory system for context retention
• Human-in-the-Loop: Approval gates for sensitive operations
• Full Observability: Distributed tracing, metrics, cost tracking
• Error Recovery: Self-healing with exponential backoff
• Cost Control: Budget limits and usage tracking
• Multiple Modes: Autonomous, Supervised, Interactive, Research

Project Structure

agent-actor/
├── .actor/
│   ├── actor.json           # Actor configuration
│   ├── input_schema.json    # Input schema with UI
│   ├── dataset_schema.json  # Output schema
│   ├── pay_per_event.json   # Monetization config
│   └── Dockerfile           # Container setup
├── src/
│   ├── main.py              # Entry point
│   ├── core/
│   │   ├── agent.py         # Main agent orchestrator
│   │   ├── state.py         # State management
│   │   └── control_flow.py  # Flow control engine
│   ├── patterns/
│   │   ├── react.py         # ReAct implementation
│   │   └── reflection.py    # Reflexion implementation
│   ├── memory/
│   │   └── memory_system.py # Multi-tier memory
│   ├── tools/
│   │   ├── instagram.py     # Instagram scraper tool
│   │   └── analysis.py      # Analysis tools
│   └── observability/
│       └── tracing.py       # Tracing & metrics
└── requirements.txt

Usage

Basic Usage

from src.core.agent import AgentBuilder, AgentMode

# Build the agent
agent = (AgentBuilder()
    .with_name("SocialMediaAnalyzer")
    .with_mode(AgentMode.SUPERVISED)
    .with_memory()
    .with_reflection()
    .with_budget(1.0)  # $1 USD limit
    .with_tools(instagram_tools)
    .with_tools(analysis_tools)
    .build(llm_client))

# Run a task
result = await agent.run(
    task="Analyze the last 10 posts from @openai and summarize AI trends"
)

print(f"Success: {result.success}")
print(f"Response: {result.output}")
print(f"Cost: ${result.cost_usd:.4f}")

Input Configuration

Parameter	Description	Default
query	The task for the agent	Required
modelName	LLM model to use	gpt-4o
mode	Operating mode	supervised
maxSteps	Maximum reasoning steps	20
budgetLimit	Cost limit in USD	None
enableMemory	Enable memory system	true
enableReflection	Enable self-reflection	true

Agent Modes

• Autonomous: Full autonomy within defined constraints
• Supervised: Requires approval for important/risky actions
• Interactive: Step-by-step execution with human guidance
• Research: Read-only mode, no side effects

Architecture Deep Dive

State Management (Factor 5 & 12)

# State is the single source of truth
# Agent is a pure function: (state, event) => new_state

class AgentState:
    events: List[AgentEvent]        # Append-only event log
    reasoning_chain: List[Step]     # ReAct reasoning trace
    conversation: ConversationCtx   # Short-term context
    reflections: List[Reflection]   # Self-improvement learnings

Control Flow (Factor 8)

# Own your control flow - deterministic routing
async def handle_next_step(thread):
    next_step = await determine_next_step(thread)

    if next_step.intent == 'request_clarification':
        # INTERRUPT: Break loop, wait for human
        await send_message_to_human(next_step)
        break

    elif next_step.intent == 'fetch_data':
        # SYNC: Execute and continue
        result = await fetch_data(next_step)
        thread.events.append(result)
        continue

    elif next_step.intent == 'deploy':
        # HIGH-RISK: Request approval
        await request_human_approval(next_step)
        break

Error Recovery (Factor 9)

consecutive_errors = 0

while True:
    try:
        result = await handle_step(thread, step)
        consecutive_errors = 0  # Reset on success
    except Exception as e:
        consecutive_errors += 1
        if consecutive_errors < 3:
            # Self-healing: add error to context for LLM
            thread.events.append({"type": "error", "data": format_error(e)})
            continue
        else:
            # Escalate to human after threshold
            await escalate_to_human()
            break

Memory System

┌─────────────────────────────────────────────────────────┐
│                    Memory System                         │
├─────────────────────────────────────────────────────────┤
│  Working Memory (Short-term)                             │
│  ├── Current goal                                        │
│  ├── Active plan                                         │
│  ├── Recent events (20 items)                           │
│  └── Entity tracking                                     │
├─────────────────────────────────────────────────────────┤
│  Episodic Memory (Medium-term)                          │
│  ├── Past task experiences                              │
│  ├── Success/failure patterns                           │
│  └── Specific event memories                            │
├─────────────────────────────────────────────────────────┤
│  Semantic Memory (Long-term)                            │
│  ├── General knowledge/facts                            │
│  ├── Consolidated learnings                             │
│  └── Entity relationships                               │
├─────────────────────────────────────────────────────────┤
│  Procedural Memory                                       │
│  ├── How to perform tasks                               │
│  ├── Tool usage patterns                                │
│  └── Strategy templates                                  │
└─────────────────────────────────────────────────────────┘

Observability

The agent includes comprehensive observability:

• Distributed Tracing: Track every step with span hierarchy
• Metrics Collection: Counters, gauges, histograms, timers
• Cost Tracking: Per-model, per-operation cost breakdown
• Structured Logging: JSON logs for aggregation

# Get execution summary
summary = agent.get_observability_summary()

# Returns:
{
    "trace": {
        "total_spans": 15,
        "total_duration_ms": 12500,
        "total_tokens": 8432,
        "total_cost_usd": 0.0847
    },
    "metrics": {
        "tool_calls_success": 5,
        "tool_calls_error": 1
    },
    "costs": {
        "cost_by_model": {"gpt-4o": 0.0847},
        "cost_by_operation": {"generate": 0.0647, "analyze": 0.02}
    }
}

Best Practices (from 12-Factor Agents)

1. Small, Focused Agents: 3-20 steps max per agent
2. Own Your Context Window: Custom formats for efficiency
3. Tools Are Structured Outputs: Function calling IS structured output
4. Contact Humans with Tools: Human interaction as first-class tool
5. Compact Errors: Self-healing with formatted error context
6. Stateless Reducer: Agent = pure function (state, event) => state

References

• ReAct Paper - Synergizing Reasoning and Acting
• Reflexion Paper - Language Agents with Verbal Reinforcement
• 12-Factor Agents - Production patterns
• LangGraph Memory - Stateful workflows
• CrewAI - Multi-agent orchestration

Development

# Install dependencies
pip install -r requirements.txt

# Run locally
apify run

# Deploy to Apify
apify push

License

MIT License