Data Engineer with Python

2 months ago


Wellington City, New Zealand Unison Consulting Pte Ltd Full time

Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
- Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
- Optimize and troubleshoot AWS Glue jobs for performance and reliability.
- Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
- Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
- Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
- Optimize ETL workflows for performance and efficiency.
- Collaborate with data architects to design and implement data models that support business requirements.
- Ensure data structures are optimized for analytics and reporting.
- Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets.
- Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs.
- Implement indexing, partitioning, and other database optimization techniques.
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
- Implement monitoring solutions to track data pipeline performance and proactively identify and address issues
- Ensure compliance with data privacy regulations and company policies.
- Stay abreast of industry trends and advancements in data engineering, Python, and PySpark.

**Requirements**:

- Proficiency in Python and PySpark.
- Strong knowledge of data engineering concepts and best practices.
- Hands-on experience with AWS Glue and other AWS services.
- Experience with big data technologies and distributed computing.
- Familiarity with database management systems (SQL and NoSQL).
- Understanding of ETL processes and data modeling.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Bachelor's degree in Computer Science, Information Technology, or a related field.


  • Python Engineer

    2 months ago


    Auckland City, New Zealand First AML Full time

    **Our Python Engineer Role and the Engineering team** We are looking for an **Intermediate Python Engineer** to join our amazing Engineering team. Problems are our passion and in this role you will use Python as a programming language to solve a variety of business problems. This role reports to our VP of Engineering and is responsible for working on...

  • Python Engineer

    2 months ago


    Auckland City, New Zealand First AML Full time

    **Introduction**: Who are we and why work for us? We've created a world-class SaaS product that automates and solves our customers due diligence problems so that they can stay focused on running their business. First AML isn't just a “start up”, we are well funded and in growth mode which for you means your career will grow with us as we create a...


  • Auckland City, New Zealand 73rd Solution Full time

    **Responsibilities**: - Write effective, high-quality Python code to solve problems - Clearly explain the reasoning behind the code in Jupyter notebooks - Debug code and produce thorough documentation - Using free datasets (such as Kaggle, the UN, the US government, etc.), use your data analytic skills to formulate and respond to important business issues -...

  • Python Engineer

    3 months ago


    Auckland City, New Zealand First AML Full time

    **Our Python Engineer Role and the Engineering team** We are looking for an** Intermediate Python Engineer** to join our amazing Engineering team. Problems are our passion and in this role you will use Python as a programming language to solve a variety of business problems. This role reports to our VP of Engineering and is responsible for working on...

  • Data Engineer

    2 months ago


    Wellington City, New Zealand Talent Army Full time

    Wellington-based hybrid working - AWS/Python/ETL Pipelines - Join a fast growing SaaS company **About the Company**: This company are a technology company serving a global market with platforms, products and services. They help their clients deliver intelligent and adaptable transport solutions and exceptional user experiences. **About the Role**: This is...

  • Engineer - Data

    4 weeks ago


    Wellington City, New Zealand Xero Full time

    Xero is a beautiful, easy-to-use platform that helps small businesses and their accounting and bookkeeping advisors grow and thrive. At Xero, our purpose is to make life better for people in small business, their advisors, and communities around the world. This purpose sits at the centre of everything we do. We support our people to do the best work of...

  • Engineer - Data

    1 day ago


    Wellington City, New Zealand Xero Full time

    Xero is a beautiful, easy-to-use platform that helps small businesses and their accounting and bookkeeping advisors grow and thrive. At Xero, our purpose is to make life better for people in small business, their advisors, and communities around the world. This purpose sits at the centre of everything we do. We support our people to do the best work of...

  • Data Engineer

    3 weeks ago


    Wellington City, New Zealand New Zealand Government Full time

    5 Weeks annual leave (and flexi-leave available) - 15 Wellness days and an annual wellness benefit - Permanent Wellington based role with flexible working options Join our dynamic team as a data engineer! As a key player in our data-driven environment, where you'll collaborate and will be able to shape our vision and impact the future of the way we work.Te...

  • Engineer - Data

    2 weeks ago


    Wellington, Wellington, New Zealand Xero Full time

    Xero is a beautiful, easy-to-use platform that helps small businesses and their accounting and bookkeeping advisors grow and thrive.At Xero, our purpose is to make life better for people in small business, their advisors, and communities around the world. This purpose sits at the centre of everything we do. We support our people to do the best work of their...

  • Data Engineer

    2 weeks ago


    Wellington City, New Zealand New Zealand Government Full time

    Data EngineerReserve Bank of New Zealand - 5 Weeks annual leave (and flexi-leave available) - 15 Wellness days and an annual wellness benefit - Permanent Wellington based role with flexible working options Te tūranga - The roleAs a Data Engineer you will work to develop, support and maintain the Bank's data-based solutions. This is an opportunity where you...

  • Data Engineer

    2 weeks ago


    Wellington City, New Zealand New Zealand Government Full time

    **Type**: PermanentHours: Full timeLocation: Wellington Business Group: Te Waka Aukaha/Planning Design and Assurance Branch: Kāwai ki te Iwi / Service Delivery and OperationsSalary: $105,332 - $141,672 We're looking for a driven and experienced Data Engineer to join our Business Intelligence & Insights team to enable advanced analytics solutions across...

  • Data Engineer

    2 weeks ago


    Wellington City, New Zealand Department of Internal Affairs NZ Full time

    **Type**:Permanent **Hours**:Full time **Location**:Wellington **Business Group**:Te Waka Aukaha/Planning Design and Assurance **Branch**:Kāwai ki te Iwi / Service Delivery and Operations **Salary**:$105,332 - $141,672 We’re looking for a driven and experienced Data Engineer to join our Business Intelligence & Insights team to enable advanced analytics...


  • Wellington City, New Zealand Zendesk Full time

    **Job Description**: Would you like to join our journey in building the next generation reporting, analytics and data platform? Our goal is to help 160,000+ global businesses truly understand customer interactions at depths never thought possible until now. Data is at the heart of Zendesk’s business! This is a great opportunity where you can have a huge...

  • Data Engineer

    2 weeks ago


    Wellington City, New Zealand New Zealand Government Full time

    **Type**: Fixed Term (3 Years)Hours: Full timeLocation: Wellington Business Group: Te Waka Aukaha/Planning Design and Assurance Branch: Kāwai ki te Iwi / Service Delivery and OperationsSalary: $105,332 - $141,672 Mō te tīma | About our Teams Te Waka Aukaha / Planning Design and Assurance Group provides a range of support and enabling services to Kāwai ki...

  • Data Engineer

    2 weeks ago


    Wellington City, New Zealand Department of Internal Affairs NZ Full time

    **Type**:Fixed Term (3 Years) **Hours**:Full time **Location**:Wellington **Business Group**:Te Waka Aukaha/Planning Design and Assurance **Branch**:Kāwai ki te Iwi / Service Delivery and Operations **Salary**:$105,332 - $141,672 We’re looking for a driven and experienced Data Engineer to join our Business Intelligence & Insights team to enable advanced...

  • Data Engineer

    2 weeks ago


    Wellington, Wellington, New Zealand New Zealand Government Full time

    Data EngineerReserve Bank of New Zealand 5 Weeks annual leave (and flexileave available) 15 Wellness days and an annual wellness benefit Permanent Wellington based role with flexible working optionsTe tūranga - The roleAs a Data Engineer you will work to develop, support and maintain the Bank's data-based solutions. This is an opportunity where you will be...


  • Auckland City, New Zealand Robert Half Full time

    **The Company** Embark on an exciting journey with an influential Kiwi enterprise that reaches across the globe. Specializing in groundbreaking data privacy solutions, this company empowers clients to secure and leverage their data assets seamlessly. Renowned for agility, advanced technology, and transformative solutions, you're invited to join a team of...

  • Data Engineer

    3 weeks ago


    Auckland City, New Zealand Robert Half Full time

    **The Company** Having been founded over 10 years ago, this stable and growing company develops a platform that is used for derivative trading with their clients being larger NZ corporates. **The Role** Reporting to the CEO, you will be working as part of a small team of three and will have responsibility for the data engineering function within the...

  • Senior Data Engineer

    3 weeks ago


    Auckland City, New Zealand Potentia Full time

    **Why You Would Like It**: - Collaborative environment where innovation thrives - Competitive remuneration package, including 8% KiwiSaver and bonus schemes - Central location with easy access and modern facilities ***The Opportunity**: Seize the chance to work amongst the best in the financial industry and use your expertise in.NET, Python, SQL, Data...

  • Data Engineer

    2 weeks ago


    Wellington Central, Wellington, New Zealand Absolute IT Recruitment Specialists Full time

    Wellington Central- Data/Data Warehousing/Business Intelligence- Market Rate- Full Time- Contracting- 30 Jun 2023ABOUT THE COMPANYOur client is a large public sector organisation, with responsibities for ensure the safety and security of New Zealand.ABOUT THE ROLEWith a large multi-agency programme of work underway, our client is looking for an experienced...