NEW JOB OPENING
SENIOR DATA ENGINEER - SPARK, SQL & DATA PIPELINE RELIABILITY
IN Menlo Park, CA, USA!

Required Education:

• Bachelor's degree in a technical field such as computer science, computer engineering or related field required.



Required Experience:

• Building and optimizing large-scale pipelines over Hive/Spark-based warehouse tables, including data modeling for analytics-ready datasets.

• Pipeline orchestration and reliability — scheduling and maintaining workflows via Dataswarm (or equivalent), handling backfills, troubleshooting failures, and implementing data quality checks and monitoring.

• Data warehousing fundamentals at scale — partitioning, incremental loads, performance tuning, and hardening/maintaining legacy pipelines for correctness and SLAs.



Preferred Licenses/Certifications:

• Process certification, such as, Six Sigma, CBPP, BPM, ISO 20000, ITIL, CMMI.



Required Qualifications:

• Ability to work as part of a team, as well as work independently or with minimal direction.

• Excellent written, presentation, and verbal communication skills.

• Collaborate with data architects, modelers and IT team members on project goals.



Preferred Qualifications:

• Familiarity with operations/support analytics domain (workforce, capacity, productivity metrics).

• Version control (Git), code review habits, and documentation discipline.

• Experience automating manual workflows/building self-serve datasets.

• Exposure to data quality frameworks, anomaly detection, or SLA alerting.



Required Technical Skills:

• Advanced SQL (Presto/Trino/Spark SQL).

• Basic Python for production ETL/ELT.

• Dataswarm (or equivalent pipeline orchestration).

• Hive/Spark-based warehouse tables.

• Strong PC skills including knowledge of Microsoft SharePoint.



Preferred Technical Skills:

• Dashboard backend experience (Unidash/Tableau or similar) supporting ops reporting.

• Git.



Job Overview:

• The main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization.

• The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization’s data assets.

• This role sits in GO Analytics – Global Response Ecosystem, the team that provides trusted data foundations for company's global operations and response organizations.



Responsibilities:

• Design, construct, install, test and maintain highly scalable data management systems.

• Ensure systems meet business requirements and industry practices.

• Design, implement, automate and maintain large scale enterprise data ETL processes.

• Build high-performance algorithms, prototypes, predictive models and proof of concepts.

• Partner with data scientists and operations stakeholders to understand data needs and translate them into reliable pipelines and tables.

• Build, maintain, and optimize ETL/ELT pipelines (SQL + Python/Spark) and orchestration schedules.

• Monitor pipeline health, troubleshoot failures, and implement data quality checks and documentation.

• Contribute to data modeling for analytics-ready datasets and support ad-hoc data requests and dashboard backends.

• **Only those lawfully authorized to work in the designated country associated with the position will be considered.**

• **Please note that all Position start dates and duration are estimates and may be reduced or lengthened based upon a client’s business needs and requirements.**