ETL Expert

Client and project description

Our customer – the Leading Supply Chain Management Platform for Cannabis and Hemp that helps track the product’s journey from harvest to packaged goods with a transparent chain of custody to improve yields and meet compliance requirements.

Operators enter the data on the production process progress, letting managers monitor activity in real-time mode. The platform provides you with visibility into every aspect of your unique production processes, ensuring everyone is on the same page and allowing you to make data-driven and informed decisions on how to best scale your operations.

We expect you to

We have a business need to make data available to customers, so they can run their own BI tools against it. We’d like to make this data available via a standalone Metabase instance as well as embedding some Metabase elements directly inside our app.

You will work with us to develop a “public schema” that reflects the denormalization expressed in the graph. The purpose of exposing a denormalized view of our data is to enable customers to more intuitively understand their data and to gain insights into it.

Our schema is complicated and heavily normalized, which makes it hard to use out-of-the-box for BI tools.
Enforcing tenant isolation, and building robust authentication and authorization, will be necessary. Our customer’s data can never be visible to another customer.

This data doesn’t have to be updated in real-time – a nightly job should be sufficient.

We already have a way to quickly construct a graph representation of a customer’s data in-memory, so a simple approach to this might be to use this graph to populate a set of tables.

Required skills

This role requires a good general background in working with relational databases, as well as specific knowledge of PostgreSQL.

As well as experience/willingness to work in a fast-moving start-up environment with lean/agile processes and possible lack of clearly defined tasks.

Requirements:

  • Strong experience with Python & Django;
  • Experience with PostgreSQL and MetaBase/Looker or similar;
  • Solid knowledge of ETL techniques;
  • Practical experience in data integration development;
  • Experience with the GCP services related/used for ETL;
  • Ability to effectively communicate challenges and issues both verbally and written;
  • Intermediate+ or higher written and spoken English, ability to communicate directly with the client;

Main areas:

  • Building data marts;
  • Processing domain data to prepare insights;
  • Preparing reusable Metabase reports;