FULL_TIME
Job Description About this Role As a Data Platform Engineer — External Data Acquisition at Astro Indonesia, you will build and run ASTRO's system for collecting data from external sources such as websites, web/mobile applications, APIs, public datasets, and third-party data providers. A major portion of this position centers on production-grade web and application data acquisition for pricing and market intelligence use cases. You will be responsible for the acquisition layer—from understanding external systems and extracting data to delivering dependable landing data for downstream processing. Focus: External Data Acquisition, Web & App Scraping, Python, HTTP/API, Data Pipelines, Reliability Responsibilities: Build and maintain dependable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources. Examine browser, mobile application, API, and network traffic to understand how data is delivered and design the suitable acquisition approach and runtime, from direct HTTP/API extraction to browser automation. Develop acquisition systems that stay dependable as external websites, applications, APIs, and data contracts change. Address real-world source complexity including sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures. Construct resilient and maintainable acquisition systems with proper testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting. Run acquisition workloads using appropriate workflow orchestration, compute, data storage/warehousing, and cloud infrastructure. Continuously enhance reliability, scalability, maintainability, and total cost of ownership, including evaluating build-vs-buy approaches for external data. Investigate new technologies and AI-assisted development or automation to improve how external-data pipelines are built, operated, and maintained. Work with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to translate external-data needs into sustainable platform capabilities. Minimum Qualifications Strong Python/software engineering fundamentals and experience building maintainable production systems. Hands-on experience building or operating web scraping, crawling, external API, or similar data acquisition systems. Solid understanding of HTTP, APIs, JSON, headers, cookies, sessions, authentication, pagination, and request/response behavior. Hands-on experience with browser/network debugging and the capability to investigate how websites or applications communicate with their backend services. Experience with both HTTP-based extraction and browser automation, using tools such as Requests/HTTPX, Selenium, Playwright, Scrapy, or equivalent. Understanding of production reliability patterns such as retry, timeout, rate limiting, concurrency, idempotency, and failure recovery. Solid grasp of SQL, data storage/warehousing, schema evolution, workflow orchestration, containerized workloads, and cloud infrastructure fundamentals. Strong troubleshooting skills, especially when dealing with frequently changing external systems outside your direct control. Good to Have Experience acquiring data from mobile applications, marketplaces, e-commerce, grocery, food-delivery, or other large consumer platforms. Experience with asynchronous or distributed acquisition systems and high-volume crawling. Familiarity with proxy infrastructure, data observability, CI/CD, and infrastructure-as-code. Experience using AI-assisted development or automation to improve data acquisition workflows. Experience designing reusable scraping frameworks or shared acquisition platforms. Experience with geospatial or other non-traditional external datasets.
Data Platform Engineer
Astro Indonesia • Jakarta Barat
Gaji tidak dicantumkan
Deskripsi Pekerjaan
Ringkasan
- Perusahaan
- Astro Indonesia
- Lokasi
- Jakarta Barat
- Tipe kerja
- FULL_TIME
- Gaji
- Gaji tidak dicantumkan
- Tanggal tayang
- 23 Sep 2026
- Terlihat sejak
- 2 Okt 2026
Sumber & keterlacakan data
JobStreet / Jobsdb (SEEK)Buka sumber asli ↗https://id.jobstreet.com/job/94836276
Siap melamar? Langsung di sumber asli.Buka Halaman Lamaran
Lowongan Terkait Lainnya
FULL_TIME1bln
Flutter Developer
PT Cashlez Worldwide Indonesia • Jakarta Pusat
Gaji tidak dicantumkanLihat Detail →
LOWONGAN1blniOS Developer
Code • Jakarta Selatan
Gaji tidak dicantumkanLihat Detail →
FULL_TIME1blnProgrammer
PT. Lestari Mulia Sentosa • Sidoarjo
Gaji tidak dicantumkanLihat Detail →
LOWONGAN1blnMpbile Programer Android
PT Masmedia Buana Pustaka • Sidoarjo
Gaji tidak dicantumkanLihat Detail →