Data Engineer

Summary

We're looking for a skilled Data Engineer with a strong data engineering background to be a part of our consulting practice. This role focuses on implementing, and optimising scalable data architectures, including data lakes, data warehouses, and data pipelines.

The ideal candidate has deep expertise in data engineering, modelling, and cloud-native platforms such as Snowflake and DBT. Experience in building robust ETL/ELT pipelines, performance tuning, and enabling data consumption for analytics and AI use cases is essential. This position requires autonomy, hands-on technical leadership, and the ability to serve as a trusted advisor on critical data projects.

Key Responsibilities

• Deploy and optimise ETL/ELT workflows to efficiently extract, transform, and load data from multiple sources into data warehouses and analytics platforms.
• Deploy and manage data lakes, data warehouses, and data marts, ensuring structured and unstructured data is efficiently stored, processed, and accessible.
• Must work with cloud platforms like AWS, Azure, or GCP, leveraging tools such as Snowflake, Redshift, and BigQuery to handle large-scale data processing.
• Design, maintain, and optimize SQL and NoSQL databases, ensuring high performance, scalability, and availability for analytics and operational use.
• Implement data validation, cleansing, and governance frameworks to maintain accuracy, consistency, security, and compliance with GDPR, HIPAA, and other regulations.
• Optimise query performance, indexing, partitioning, and caching while automating data workflows through CI/CD, Infrastructure as Code (IaC), and monitoring solutions.
• Work closely with BI team to ensure data is well-structured and optimized for reporting and dashboards in Power BI, Tableau, Looker, and Qlik.
• Partner with Data Scientists, Analysts, and Business Teams to enable data-driven decision-making, advanced analytics, and self-service BI capabilities.
• Translate architecture and designs to engineer solutions to host proof-of-concepts, prototypes, non-production and production solutions.
• Collaborate with designers and product managers to understand user requirements and translate them into technical solutions.
• Help define options papers for solutions and platforms.
• Fulfil all legislative and company policy requirements.
• Have active involvement in health and safety in the workplace.

Key Requirements

• Five years of experience working as a Data Engineer, ideally in the consultancy space.
• Strong expertise in
  Snowflake
  : including schema design, performance tuning, cost management, and integration with other tools and platforms. Strong experience with
  dbt
  for transforming raw data into clean, structured datasets ready for analysis and reporting. Proficiency in SQL, with experience writing complex queries and optimizing for performance.
• Experience with new features of Snowflake such as Snowflake Horizon will be looked upon favourably.
• Three plus years of experience in data engineering, ETL processes, and building data pipelines.
• Strong knowledge of data modeling, database design, and data warehousing.
• Hands-on experience with Azure Data Services (e.g., Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage).
• Hands-on experience in working with Snowflake for data warehousing, data sharing, and cloud-based data architecture.
• Experience with AWS Data Services (e.g., Redshift, S3, Glue) would be a plus.
• Proficiency in building ETL/ELT pipelines using tools like AWS Glue, Azure Data Factory, and DBT for data transformation and integration.
• Experience with CI/CD pipelines for automating data workflows.
• Strong experience in automating data workflows and orchestrating data pipelines in cloud environments.
• Experience implementing data governance strategies and security protocols in cloud environments, including encryption, access control, and auditing in Snowflake, AWS, and Azure.
• Experience with real-time data streaming and processing using Azure Stream Analytics, or Snowflake's Snowpipe for automated data loading.
• Proficiency in SQL, Python, Scala, or other programming languages for data processing.
• Ability to integrate data from multiple sources, both structured and unstructured.
• Strong understanding of data governance, data quality, and security best practices.
• Strong ability to work with cross functional teams, including engineers, analysts, and business leaders.
• Excellent verbal and written communication skills to explain technical concepts clearly.
• Passion for emerging technologies, data architecture advancements, and innovation in the field.
• A pragmatic thinker who aligns with our values, avoiding over-engineering solutions and reviewing the current state before proposing solutions.
• You must demonstrate versatility in learning technologies to be technology agnostic based on demand.
• Must have an always learning attitude.
• Growth mindset, positive attitude & strong interest in solving client challenges, adapting to a changing work environment, and dealing with new issues.
• Certifications from Snowflake would be a plus.
• Must be based in Auckland or Wellington.
• Travel to the customer site (on-demand).

Here's a Taste of What's on Offer at dataengine:

• Opportunity to solve real-world business problems.
• Collaborative and supportive work environment.
• Competitive salary.
• Be at the forefront of data science innovation in a dynamic startup environment.
• Work on a variety of challenging and impactful projects across different industries.
• Opportunity to learn and grow your skillset with the latest technologies.
• Continuous learning and development opportunities.
• Weekly fresh fruit, snacks, monthly industry led lunches and active staff activities.
• Be part of a company that is shaping the future of data-driven solutions.

About dataengine:

dataengine, a New Zealand-based leader in data solutions, empowers businesses to unlock the hidden value within their data and achieve true transformation.
We connect people, processes, and technology
for optimised efficiency and sustainable impact.
Founded in 2018, we've grown rapidly thanks to our exceptional team and commitment to delivering
practical, impactful results
. We believe in
building the foundation right
, starting with strategic planning and robust architecture.

What sets us apart?

• Experienced & Pragmatic:
  Our team boasts extensive expertise, focusing on
  tangible outcomes, not just presentations
  .
• Small Wins, Big Impact:
  We deliver
  incremental value drops
  , ensuring our customers see results quickly and continuously.
• Holistic Enablement:
  We develop
  with our customers, in their environment
  , empowering our team to own and optimise solutions.

Leading the way in Generative AI:
We've developed a unique
standalone AI engine
, providing secure and accessible tools for our customers' AI journey. From
use case brainstorming to model deployment and tuning
, we offer comprehensive support throughout the entire lifecycle.

Beyond AI:
Our services range from
advisory and resourcing
to
full-scale platform design and build
, catering to any data-driven business need.

Our Values

Nimble
- We move fast but think big

Quality
- We do it right the first time

Co-Creation:
We work together to solve precise problems

We are All Ears
- We listen and then do

Progressive Thinkers
- We are creators and doers

Pragmatic
– We don't over-engineer