Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake)

Are you a seasoned Data Engineer passionate about crafting cutting-edge lakehouse architectures and driving innovation in data sharing? If you thrive on tackling complex data challenges, possess a deep command of GCP, Python, and the latest open table formats like Iceberg and Delta Lake, then Railroad19 has an exceptional opportunity for you. We are searching for a Senior Data Engineer to join our "A" team, someone ready to design and implement robust, reusable data adapters that bridge BigQuery products with platforms like Snowflake and Databricks. This is your chance to make a significant impact, working remotely from anywhere in the world, collaborating with some of the best in the business.

Overview

Railroad19 Inc. is a dynamic software development company specializing in custom solutions and expert services. We pride ourselves on assembling an "A" team of developers and architects who are driven by a passion for excellence and a commitment to leading the industry. Our culture fosters innovation, hard work, and mutual respect, creating an environment where your contributions are truly valued.

This is a 100% remote, full-time position open to exceptional talent from around the globe. While our headquarters are in Saratoga Springs, New York, we believe the best talent knows no geographical bounds. We offer competitive compensation and a comprehensive benefits package including Medical, Dental, Vision, and Pet Insurance, along with Paid Time Off and a 401(k) plan. Please note, we are seeking full-time employees exclusively; no 1099, C2C, or Corp-to-Corp arrangements will be considered, and agencies are asked not to contact us.

Key Responsibilities

Architect, design, and implement robust, scalable data lakehouse solutions on the Google Cloud Platform (GCP).
Develop and deploy advanced data processing pipelines using Python and Apache Spark.
Create reusable data-sharing adapters to facilitate seamless integration between BigQuery-backed data products and platforms like Snowflake and Databricks.
Leverage modern open table formats such as Apache Iceberg (including UniForm) and Delta Lake for efficient data management and interoperability.
Design and build real-time data ingestion and streaming solutions, potentially utilizing Apache Kafka.
Ensure data quality, integrity, and security across all implemented data solutions.
Collaborate closely with product managers, data scientists, and other engineering teams to understand data requirements and deliver impactful solutions.
Mentor junior engineers and contribute to best practices within the data engineering team.
Stay abreast of emerging technologies and industry trends in the data engineering space, particularly within the GCP ecosystem.

Requirements

Extensive hands-on experience as a Senior Data Engineer, specifically in designing and building modern data lakehouse architectures.
Deep expertise with Google Cloud Platform (GCP) services, particularly BigQuery, Dataflow, Dataproc, Cloud Storage, and other relevant data services.
Strong proficiency in Python development for data engineering, including experience with data manipulation libraries and frameworks like Spark.
Proven experience with open table formats such as Apache Iceberg and Delta Lake.
Familiarity with data warehousing solutions like Snowflake and experience integrating with platforms like Databricks.
Experience with real-time data streaming technologies, ideally Apache Kafka.
Solid understanding of SQL and database concepts.
Demonstrated ability to design and implement scalable, resilient, and secure data pipelines.
Excellent problem-solving skills and the ability to work independently in a remote environment.
Strong communication and collaboration skills to effectively work with cross-functional teams.
A Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

What You'll Gain

The opportunity to work with cutting-edge data technologies and contribute to innovative lakehouse architectures.
Be part of Railroad19's "A" team, collaborating with highly skilled and passionate professionals.
A supportive and engaging remote work environment that values your expertise and provides the resources for your success.
Competitive compensation paired with a comprehensive benefits package, including Medical, Dental, Vision, and Pet Insurance, Paid Time Off, and a 401(k) plan.
Significant impact on key data initiatives and the chance to shape the future of data sharing within our solutions.
Continuous learning and professional growth opportunities within a forward-thinking company.