Principal Data Engineer Platform

Location

New York City, New York, United States

Salary

60000 - 100000 a year (US Dollars)

Description

The Role

LeafLink is seeking a Principal Data Engineer to join our remote-friendly team, headquartered in NYC, who is passionate about working with teams that solve interesting, large-scale problems rapidly. This impactful position enables LeafLink to coordinate and integrate with 3rd party data sets and proprietary data to produce valuable insights into business and customer needs. As a member of our engineering team, you will be in a position to have a direct and lasting impact everywhere in the company. Your contribution will be immediate and have positive ripple effects across not just our business, but also the business of each of our customers.

LeafLink is currently tackling a large-scale platform overhaul that will strengthen our position as a technical leader within the industry. As such, this role has the opportunity to help lead, shape, and grow the data and machine learning architecture within our platform, as well as work with new and growing technologies. It’s a very exciting time to join our engineering team!

Ideal candidates for this position should possess a keen mind for solving tough problems with the ideal solution, partnering effectively with various team members along the way. They should be deeply passionate about organizing and managing data at scale for various use cases. They should be personable, efficient, flexible, and communicative, have a strong desire to implement change, grow, mature, and have a passion and love for their work. This role comes with the opportunity to be a high performer within a fast-paced, dynamic, and quickly growing department in all areas.

What You’ll Be Doing

Audit, design, and maintain a high-performing, modular, and optimal data pipeline architecture for structured and unstructured use cases around machine learning, reporting, and analytics
Design and co-build with Cloud and DevOps the infrastructure and operations required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, python, and AWS cloud technologies
Keep up to date on modern technologies and trends and advocate for their inclusion within products when it makes sense
Analyze and evaluate existing solutions and make decisions on whether to extend or refactor as needed with a major focus on improving our pipeline and reporting performance
Work with the CTO and department stakeholders to properly plan short and long-term goals, and define and execute a technical roadmap that continues to evolve LeafLink’s data capabilities and functionality to meet the needs of our Business and Product Vision.
Work collaboratively with multiple cross-functional agile teams to help deliver end-to-end products and features enabled by our data pipeline, seeing them through from conception to delivery
Help define, document, evolve, and evangelize high engineering standards, best practices, tenants, and data management & governance across data and analytics engineering
Move quickly and intelligently - seeing technical debt as your nemesis and eliminating risk
Effectively communicate the complexity of your work to technical and non-technical audiences through non-written and written mediums
Design, develop, and test data models in our data warehouse that enable data and analytics processes
Help define and build our enterprise data catalog and dictionary
Troubleshoot, diagnose and address data quality issues quickly and effectively while implementing solutions to combat this at scale, including improved quality controls and observability and monitoring
Provide mentorship and growth to our BE and Data engineers while creating repeatable and scalable solutions and patterns

What You’ll Bring to the Team

Minimum of 10 years experience in a professional working environment on a data or engineering team
Advanced working SQL knowledge and experience working with relational and non-relational databases, query authoring (SQL) as well as working familiarity with a variety of data stores
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Expertise writing Python processing jobs to ingest a variety of structured and unstructured data received from various sources & formats such as Rest APIs, Flat Files, and Logs with the ability to support and scale to both smaller and larger dataset ingestions
They should also have experience using the following software/tools:

Experience with object-oriented/object function scripting in Python and data processing libraries such as requests, pandas, sqlalchemy
Experience with relational SQL and NoSQL databases, such as Redshift or comparable cloud-based OLAP databases such as Snowflake
Experience with data pipeline and workflow management tools: Airflow
Experience with AWS cloud services
Hands-on experience with technologies such as Dynamo, Terraform, Kubernetes, Fivetran, and dbt is a strong plus
Experience with designing and implementing machine learning enablement tools and infrastructure
Experience leveraging API-based LLM models, dynamic prompt generation, fine-tuning

Comfortable working in a fast-paced growth business with many collaborators and quickly evolving business needs
Individual contributor leadership to our data and analytics engineers and specialization on our current Platform Engineering team around data enterprise architecture and best practices
Consistency and standards to how we visualize and use our enterprise data at LeafLink through helping us define our first Data Dictionary and Catalog

LeafLink Perks & Benefits

Flexible PTO - you’re going to be working hard so enjoy time off with no cap!
A robust stock option plan to give our employees a direct stake in LeafLink’s success
5 Days of Volunteer Time Off (VTO) - giving back is important to us and we want our employees to prioritize cultivating a better community
Competitive compensation and 401k match
Comprehensive health coverage (medical, dental, vision)
Commuter Benefits through our Flexible Spending Account

LeafLink’s employee-centric culture has earned us a coveted spot on BuiltInNYC’s Best Places to Work for in 2021 list. Learn more about LeafLink’s history and the path to our First Billion in Wholesale Cannabis Orders here.

Please mention the word **AFFIRM** and tag RMzQuODYuMTYzLjE1Mg== when applying to show you read the job post completely (#RMzQuODYuMTYzLjE1Mg==). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.