Descripción del puesto

They're looking for a Senior Data Engineer to join the Data Science & Management team. The role focuses on expanding and optimizing data architecture and pipelines, as well as improving data flow and collection across cross-functional teams.

The ideal candidate has strong experience building data pipelines and data models from scratch, and enjoys optimizing data systems. This person will collaborate with product owners, analysts, and data scientists to ensure efficient and consistent data delivery.

The main project involves migrating approximately 1,000 pipelines to Snowflake, so experience with Active Batch, Talend, Snowflake, Docker, and GitHub is required. The candidate should be excited about designing and maintaining modern data architectures in an agile environment.

Responsibilities:

• Require the application of theoretical, domain/technology-specific knowledge, typically gained through formal education or relevant expertise within professional areas (e.g., engineering, software design, systems architecture) to achieve results.
• May be required to guide and influence others; however, the primary focus of the role is the application of technical and domain expertise.
• Have wide-ranging experience and use professional concepts and company objectives to resolve complex problems.
• Engage with business stakeholders to establish clear needs and connect them to solutions, including setting up prototypes and involving multiple parties in design sessions.
• Exercise judgment in selecting methods and evaluation criteria to obtain results.
• Create data collection, extraction, and transformation frameworks for structured and unstructured data.
• Develop and maintain data infrastructure systems (e.g., data warehouses), including data access points.
• Prepare and manipulate data using a variety of data integration and pipeline tools including, but not limited to, Talend, SQL, Snowflake, Kafka, and Azure Data Factory.
• Create efficient load processes, including logging, exception handling, support notification, and visibility operations.
• Organize data into formats and structures that optimize reuse and efficient delivery to business units, analytics teams, and system applications.
• Integrate data across data lakes, data warehouses, and systems applications to ensure consistent information delivery across the enterprise.
• Ensure efficient architecture and systems design.
• Build and evolve the data service layer and engage the team to assemble components for a best-in-class customer offering.
• Assess overall data architecture and integrations, and drive ongoing improvements to the solution offering.
• Lead the architecture, design, and implementation of complex data architecture and integration solutions, applying best practices throughout the full development lifecycle, including coding standards, code reviews, source control, build processes, testing, and operations.
• Collaborate with data governance and strategy teams to ensure data lineage is clearly defined and constructed to highlight data reuse and simplicity.
• Assess new opportunities to simplify data operations using new tools, technologies, file storage methods, and processes. Use team context and experience to evaluate these opportunities and bring them forward for review and implementation.

Requisitos

• Bachelor’s degree in Software Engineering, Computer Science, or a related field required; Master’s degree considered an asset. Equivalent work experience in a technology or business environment will also be considered.
• Minimum of 7 years of experience in data engineering, with a focus on structured work processes.
• Minimum of 7 years of experience developing integration solutions using data integration tools such as Talend or Azure Data Factory.
• Minimum of 3 years of experience working with real-time data streaming tools such as Kafka or equivalent.
• Experience with cloud platforms, including Azure, AWS, or GCP.
• Experience with data warehousing platforms such as Snowflake or Databricks.
• High proficiency in SQL, Python, and other programming languages.
• Strong expertise in data management, data governance, data design, and database architecture. Proven ability to manipulate, process, and extract value from large, disconnected datasets.
• Proficiency in data modeling and data architecture; experience with WhereScape RED/3D is an asset.
• Experience with big data tools such as Hadoop, Spark, and Kafka.
• Experience with both relational and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools such as Azkaban, Luigi, or Airflow.
• Experience with AWS services including EC2, EMR, RDS, and Redshift.
• Experience with stream-processing systems such as Storm or Spark Streaming.
• Proficiency in object-oriented or functional scripting languages such as Python, Java, C++, or Scala.
• Strong expertise in data modeling, data integration, data orchestration, and supporting methodologies.
• Proven experience leading large-scale projects or significant project components, with strong communication skills to engage both technical and non-technical stakeholders.
• Proficiency in multiple programming languages with the ability to design and engineer moderately complex enterprise solutions.
• Working knowledge of message queuing, stream processing, and highly scalable big data storage systems.
• Strong project management and organizational skills.

Beneficios

• Salary in USD.
• 10 paid time off (PTO) days per year.
• International certifications and trainings of your choice.
• Access to coworking spaces.
• English classes to enhance your language skills.