Software Engineer (Data Platform)
Black Crow AI
Posted on Friday, August 18, 2023
The company + vision.
Everyone’s got the message: machine learning is changing everything. And for Fortune 500 brands in every vertical, it’s actually true — ML predictions are solving billion-dollar, migraine problems. But for smaller brands, the talent and investment needed to build an ML platform are simply out of reach.
Black Crow is changing that--fast. Founded by a team that has spent the last decade in machine learning, our mission is to help DTC brands turbocharge their growth by delivering bespoke, Fortune 500-grade ML predictions. It works out of the box, delivered as a service, and costs less than one data scientist. Integration: one click.
12 months ago, we had one DTC brand using our predictions. Now we have over 150 — including SAKARA, Burrow, Kinder Beauty, and Native. There are 50,000 more who haven’t heard of us yet.
Black Crow AI recently concluded a $25 million Series A led by top investors in the DTC and B2B SaaS sectors, and is well funded for multiple years
The team + culture.
This is the 3rd company Black Crow’s founders have started together. Both previous companies scaled to >$100MM and were voted ‘Best Places to Work’ in NYC many times.
The founders are building Black Crow to be commercially successful, high growth, and valuable. But that’s insufficient as an end--we’re also building a company that we actually want to work at! That means we are human first--we don’t expect anyone to transform themselves to show up at work. It means working together is a positive force in our lives, not something we tolerate for the paycheck. And it means we see diversity as a strength.
Where you come in:
Black Crow AI is looking for a software engineer that can work on our machine learning platform - from data ingestion to model building. You'll be part of a small, close-knit team of well-seasoned engineers with a remit at the heart of our organization.
In our engineering work, we balance cutting-edge technologies with industry-proven practices. That polyglot approach provides exposure to and advancement in a variety of languages, libraries, and databases.
Our agile way of working has evolved over many years. We believe that engineers are most effective when they are involved with the whole lifecycle of software development: conceiving new features, designing and coding the change, testing, deploying, and support. Don't worry if you don't know how to do everything at first - you'll learn the essentials through pair programming with other experienced engineers on the team.
What you'll do:
- Collaborate closely with cross-functional teams, including data scientists, analysts, and other engineers, to ensure seamless integration of data solutions into our products and services.
- Produce quality code and documentation.
- Design, develop, and maintain scalable and efficient data pipelines to extract, transform, and load (ETL) data from diverse sources into our data platform.
- Utilize your expertise in data management to architect and implement data models that support both analytical and operational use cases.
- Work on data orchestration tasks, ensuring the smooth flow of data across various systems and components of our data ecosystem.
- Troubleshoot and optimize data pipelines for performance, reliability, and scalability.
- Contribute to the continuous improvement of our data engineering processes and tools, and actively participate in code reviews and design discussions.
- Maintain the confidentiality, integrity, and availability of our systems and processes in compliance with our internal policies on information security and privacy.
- Responsible for the development and deployment of secure code.
- Black Crow AI is looking for a bright and ambitious Software Engineer to bring a fresh perspective to our team.
- Minimum of 3 years of professional experience as a Software Engineer, with a focus on data engineering, data platforms, and/or ML platform.
- Proficiency in at least one programming language (e.g., Python, Java, Scala) commonly used in data engineering tasks.
- Demonstrated experience with all aspects of delivering working software, including analysis, design, automated testing, continuous integration, and continuous deployment.
- Proven ability to build and deploy large-scale data pipelines supporting mission-critical production use cases.
- Experience working with cloud-based data solutions, such as AWS, GCP, or Azure.
- Experience with working on ETL and pipelines.
- Hands-on experience with data management and modeling techniques, ensuring data integrity and quality.
- Familiarity with data orchestration tools and frameworks (e.g., Apache Airflow, Luigi, Prefect).
- Experience with data warehousing solutions and proficient SQL knowledge (e.g. Snowflake, Amazon Redshift, Google BigQuery)
- Excellent problem-solving skills and the ability to work in a collaborative team environment.
- Bachelor’s degree in computer science, software engineering, or any other engineering field.
- You can work Eastern time hours.
Nice to have:
- Experience with Terraform
- Experience with dbt
- Experience with open-source ETL frameworks like Singer, Meltano
- Experience with machine learning and machine learning frameworks/libraries (e.g. Apache Spark ML, Sci-kit learn, AWS SageMaker, etc)
- Experience developing, monitoring, and maintaining distributed systems & frameworks
- Expertise with AWS specifically (e.g. AWS Lambda, Kinesis, API Gateway, ECS, etc.)
More about us.
+ Virtual-first and welcome employees from anywhere
+ Unlimited paid time off
+ Fridays are yours - complete flexibility on if or how much you focus on Black Crow each Friday
Love your job.
We love our work and our team. Our machine learning helps ‘flick the lights on’ for companies so they see their customers more clearly and can uncover the true potential of their tools and teams. In turn, we strive to do the same for our people. We’re here to shine light on each other and unlock known (and sometimes unexpected) potential.
Our vibrant team accomplishes buckets of awesome every day. We also partake in our fair share of daily laughter. Come build the company you’ve always wanted to work for - with inspiring products and a community of passionate people.