arXiv Research Paper Normalizer 🔬

Search 2.5M+ arXiv research papers by keyword, author, or category. Returns cleaned, normalized JSON with standardized author names, unified affiliations, mapped categories, and extracted AI/ML keywords — not raw XML.

Built for AI/ML research monitoring, literature reviews, RAG dataset creation, and competitive intelligence.

✨ What Makes This Different

• XML → Clean JSON — arXiv returns Atom XML; we parse and normalize it into flat, queryable JSON
• Author Name Normalization — Consistent formatting, deduplication-ready
• Affiliation Standardization — Maps "Google Research", "Google Brain", "Google DeepMind" → canonical names
• Category Mapping — Short codes (cs.AI) expanded to full names (Artificial Intelligence) with group classification
• Keyword Extraction — NLP heuristics extract key concepts from abstracts (transformer, diffusion, GAN, RL, etc.)
• Structured Output — Dataset items + human-readable summary table + machine-readable stats

📥 Input

Field	Type	Default	Description
searchQuery	string	(required)	arXiv search query. Supports AND, OR, ANDNOT. Use au:lastname for author search.
maxResults	integer	50	Max papers to return (1–1000)
categoryFilter	string	—	Filter by category, e.g. cs.AI, cs.CL, stat.ML
sortBy	select	relevance	relevance, lastUpdatedDate, or submittedDate
dateFrom	string	—	Earliest date (YYYY-MM-DD)
dateTo	string	—	Latest date (YYYY-MM-DD)
extractKeywords	boolean	true	Extract AI/ML keywords from abstracts

Example Input

{
  "searchQuery": "transformer AND attention",
  "maxResults": 50,
  "categoryFilter": "cs.AI",
  "sortBy": "submittedDate",
  "extractKeywords": true
}

📤 Output

Each paper is a JSON object in the dataset:

{
  "title": "Attention Is All You Need",
  "authors": ["Ashish Vaswani", "Noam Shazeer", "Niki Parmar", "Jakob Uszkoreit", "Llion Jones", "Aidan N. Gomez", "Łukasz Kaiser", "Illia Polosukhin"],
  "affiliations": ["Google"],
  "categories": [
    {"code": "cs.CL", "group": "cs", "name": "Computation and Language (NLP)"},
    {"code": "cs.LG", "group": "cs", "name": "Machine Learning"}
  ],
  "categoryCodes": ["cs.CL", "cs.LG"],
  "categoryGroups": ["cs"],
  "year": "2017",
  "published": "2017-06-12T17:43:57Z",
  "updated": "2017-12-06T18:04:40Z",
  "abstract": "The dominant sequence transduction models...",
  "keywords": ["attention mechanism", "self-attention", "transformer"],
  "doi": "10.48550/arXiv.1706.03762",
  "arxivId": "1706.03762v7",
  "pdfUrl": "https://arxiv.org/pdf/1706.03762v7",
  "journalRef": "NeurIPS 2017",
  "comment": "15 pages, 5 figures"
}

🎯 Use Cases

• AI/ML Research Monitoring — Track new papers in your field daily
• Literature Reviews — Search, filter, and export normalized paper data
• RAG Datasets — Build knowledge bases from arXiv papers with clean metadata
• Competitive Intelligence — Monitor what labs and companies are publishing
• Academic Analytics — Analyze publication trends, collaboration networks, keyword evolution

🔌 API & Data Source

This actor uses the official arXiv API (Atom XML endpoint). The API is free, requires no key, and provides access to all 2.5M+ papers on arXiv. Please observe arXiv's rate limiting guidelines — we include polite delays between requests.

🏷️ Pricing

$2.00 per 1,000 results. Pay only for what you use.

---

Built by Chris The Dev — scraping and normalizing public data so you don't have to.