Mô tả công việc
The Position
At OPSWAT, we’re building cutting‑edge solutions that protect the world’s critical infrastructure. You will play a pivotal role in shaping the future of our OESIS Framework platform with cloud solution integrations. This role owns everything that happens to device compliance data after it's retrieved from a vendor API: bulk and incremental querying at scale, normalizing wildly different vendor data models into one common schema, and storing it in a way that's fast to query and safe for customers.
What you will be doing
- Build scalable ingestion pipelines that query large volumes of device/compliance data from Endpoint Device Management Platforms (such as CrowdStrike, Intune, JAMF) — both full bulk syncs and incremental/delta syncs
- Design a normalized compliance data schema that maps each vendor's distinct data model (e.g., CrowdStrike host/device telemetry, Intune compliance policy state, JAMF inventory records) into one common internal representation
- Build and tune the storage layer (relational, NoSQL, or warehouse) for high‑volume, multi‑tenant compliance data with efficient read/query patterns for downstream consumers
- Implement caching, deduplication, and delta‑sync logic that minimizes redundant API calls and respects each vendor's rate limits and query quotas
- Build ETL/ELT orchestration for scheduled and event‑driven syncs, with monitoring and alerting on pipeline health and sync failures
- Partner with the Integration Engineer on the shape of the raw API responses being handed off, and with the Auth Engineer to handle token expiry gracefully mid‑query
What we need from you
- 3+ years in data engineering: ETL/ELT pipeline design, data modeling, and large‑scale ingestion
- Strong SQL plus hands‑on experience with both relational (Postgres/MySQL) and NoSQL or columnar stores
- Experience with pipeline orchestration tools (Airflow, Dagster, Prefect) and/or streaming platforms (Kafka, Kinesis)
- Direct experience handling REST API pagination, bulk/batch queries, and rate‑limited data collection at scale
- Demonstrated experience normalizing heterogeneous schemas from multiple external data sources into one model
- Strong Python (or equivalent) for pipeline development
- Understanding of multi‑tenant data isolation and relevant compliance requirements (SOC 2, GDPR)
It would be nice if you have
- Prior experience specifically with security or compliance data (device posture and regulatory compliance (PCI‑DSS, SOX, HIPPA, etc.), vulnerability, patch management)
- Experience with time‑series or event‑driven data models for compliance state changes
- Direct familiarity with CrowdStrike, Intune, or JAMF data structures and cloud based device management frameworks
- Familiarity with Zero Trust Network Access (ZTNA) and device posture or compliance
Why join us?
At OPSWAT, we believe in empowering our people to do their best work. We encourage you to watch the video of our companies’ more than 20 years’ experience of securing critical infrastructure. You’ll be part of a mission‑driven OESIS team that values innovation, collaboration, and continuous development. Together, we’re building SDK technology that makes a difference