Mô tả công việc
Position Summary, Responsibilities and Expectations
- Design, develop, and optimize scalable ETL/ELT pipelines for enterprise data ingestion, integration, and transformation.
- Build and maintain data engineering workflows using Databricks, dbt, PySpark, SQL, and Python.
- Develop Data Lake, Data Warehouse, and Lakehouse solutions, including data modeling and incremental processing.
- Collaborate with business stakeholders to understand requirements, business processes, and challenges, and translate them into effective data and analytics solutions.
- Proactively communicate with business users and technical teams to clarify requirements, provide project updates, address issues, and ensure solutions meet business expectations.
- Ensure data quality, consistency, security, and governance through validation and automated testing.
- Prepare and integrate enterprise data to support BI, AI/ML applications, and advanced analytics.
- Develop AI/LLM solutions, including RAG, AI agents, API integrations, and intelligent workflow automation.
- Monitor and optimize data pipeline performance, reliability, resource utilization, and costs.
- Implement data orchestration, CI/CD, automated deployment, and monitoring practices.
- Collaborate with BI, AI, IT, and business teams to support solution delivery, user testing, documentation, and continuous improvement.
Essential Skills and Experience
- 2–4 years of hands‑on experience in Data Engineering, ETL/ELT, and enterprise data platforms.
- Strong SQL and Python skills, with practical experience in Databricks, PySpark, Delta Lake, and dbt.
- Experience designing, developing, and optimizing production data pipelines, data models, and Lakehouse/Data Warehouse solutions.
- Ability to understand business processes, gather and clarify requirements, and translate business needs into actionable technical solutions.
- Strong verbal and written communication skills, including the ability to explain technical concepts to non‑technical stakeholders.
- Strong analytical, problem‑solving, and interpersonal skills, with a proactive attitude and ability to collaborate across business and technical teams.
- Familiarity with cloud data platforms (Azure/Microsoft Fabric preferred), Git, CI/CD, and data quality practices.
- Understanding of AI/LLM technologies, API integrations, and AI‑driven data processing or automation.
- Nice‑to‑Have: Experience with RAG, AI agents, vector databases, Power BI, or enterprise ERP systems.
Yêu cầu kinh nghiệm
2-4 years data engineering experience; strong SQL and Python; Databricks, PySpark, Delta Lake, dbt; cloud platforms (Azure/Microsoft Fabric preferred); Git, CI/CD; AI/LLM knowledge; communication and analytical skills.