Prompt Pilot 🤖🌐

📌 Introduction

🎯 What is this Prompt Pilot?

The Prompt Pilot is a powerful AI-powered tool designed to automate web browser interactions using natural language prompts. It leverages OpenAI models to understand your instructions and perform complex browser automation tasks without writing traditional automation scripts.

🚀 Key Features

• AI-powered automation - Use natural language prompts to control the browser
• Multi-step workflows - Execute complex automation sequences with multiple steps
• Variable support - Use variables in prompts for dynamic interactions (e.g., %searchTerm%, %productName%)
• Smart waiting - Automatically wait for elements to appear after actions
• Result extraction - Extract HTML content from specific page elements
• Flexible configuration - Customize model, selectors, and timeouts

🔍 Use Cases

• Web testing - Automate testing workflows with natural language
• Data extraction - Navigate complex websites and extract structured data
• Research automation - Automate repetitive research tasks across multiple websites

---

📥 Inputs

📝 What does the actor take in?

The Prompt Pilot accepts the following inputs:

• url (required): The URL to navigate to and automate.
• steps (required): An array of automation steps to execute. Each step contains:
  • prompt (required): The natural language action prompt (e.g., "click the search button", "fill in the search field with %searchTerm%")
  • variables (optional): An object with variables to use in the prompt (e.g., { "searchTerm": "web scraping", "productName": "laptop" })
  • waitFor (optional): Configuration to wait for an element after the action:
    • selector: CSS selector or Playwright selector to wait for
    • timeout: Timeout in milliseconds (default: 20000)
• resultSelector (optional): CSS selector for the final result element to extract HTML from (default: "body").

🔒 Restrictions

• You must provide a valid URL starting with http:// or https://.
• At least one step must be provided in the steps array.
• The website must be accessible and publicly available.

✅ Example Input

{
    "url": "https://news.google.com",
    "steps": [
        {
            "prompt": "click the Top stories",
            "waitFor": {
                "selector": "h1",
                "timeout": 20000
            }
        },
        {
            "prompt": "scroll down to see more articles"
        }
    ],
    "resultSelector": "h1"
}

✅ Example Input with Variables

{
    "url": "https://example.com/search",
    "steps": [
        {
            "prompt": "fill in the search field with %searchTerm%",
            "variables": {
                "searchTerm": "web automation"
            }
        },
        {
            "prompt": "click the search button",
            "waitFor": {
                "selector": ".search-results",
                "timeout": 15000
            }
        }
    ],
    "resultSelector": ".search-results"
}

---

📤 Outputs

📝 What does the actor return?

The actor produces structured JSON output with the following fields:

• url: The URL that was automated
• resultHTML: HTML content of the result element (extracted using resultSelector)
• timestamp: ISO timestamp when the automation completed

⚠️ Restrictions

• If the resultSelector element is not found, resultHTML will be null.
• If the automation fails at any step, an error will be thrown.

✅ Example Outputs

🔹 Basic Output

{
    "url": "https://news.google.com",
    "resultHTML": "<h1>Top stories</h1>",
    "timestamp": "2025-01-27T10:30:00.000Z"
}

🔹 Output with Extracted Content

{
    "url": "https://example.com/search",
    "resultHTML": "<div class=\"search-results\"><h2>Search Results</h2><p>Found 10 results...</p></div>",
    "timestamp": "2025-01-27T10:35:15.123Z"
}

---

⚙️ How to Use

🏁 Running the Actor

1. Log in or Sign up on Apify.
2. Create a new task and select "Prompt Pilot".
3. Enter the target URL in the url field.
4. Define your automation steps in the steps array:
   • Add natural language prompts for each action
   • Optionally add variables and wait conditions
5. Set the result selector (optional) to extract specific content.
6. Start the task and receive your results in JSON format.

📊 Interpreting Results

• The resultHTML field contains the HTML content of the element specified by resultSelector.
• The timestamp field shows when the automation completed successfully.
• If resultHTML is null, the selector was not found on the page.

💡 Tips for Writing Prompts

• Be specific: Instead of "click button", use "click the search button" or "click the button with text 'Submit'"
• Use variables: For dynamic values, use %variableName% in prompts and provide values in the variables object
• Add wait conditions: Use waitFor to ensure elements appear before proceeding to the next step
• Break down complex tasks: Split complex workflows into multiple simple steps

---

🛠️ Troubleshooting

❌ Common Issues & Fixes

Issue	Possible Cause	Solution
Automation fails	Invalid prompt or element not found	Make prompts more specific, add wait conditions
Timeout error	Element didn't appear in time	Increase timeout in waitFor configuration
Empty resultHTML	Selector not found	Verify the selector exists on the page, check browser console

🔍 Debugging Tips

• Check logs: Review the actor logs to see what actions were performed
• Test selectors: Verify CSS selectors work in browser DevTools before using them
• Start simple: Begin with basic prompts and gradually add complexity
• Use wait conditions: Always wait for elements after navigation or dynamic content loading

---

🔗 Additional Resources

• Apify Platform Documentation
• OpenAI API Documentation
• Playwright Documentation

---

🙌 Acknowledgments

This actor was built with the support of the Apify community and OpenAI. Special thanks to all contributors! 🎉