Data Engineer

The Role

We are looking for a Data Engineer with backend experience to operate and extend Helda's data pipelines as we onboard healthcare organisations onto the platform. You will be the person who takes a new hospital's raw data — however it arrives — and gets it flowing cleanly into Supabase so the analytics platform can serve it.

You will work closely with healthcare org contacts (who provide the data), the senior data engineer (who builds the infrastructure you operate), and the backend engineer (who depends on clean, schema-compliant data for analytics accuracy).

What You Will Do

Healthcare Org Onboarding

• Lead the technical side of onboarding new healthcare organisations: receive their data exports, assess data quality, map their columns to Helda's canonical schemas, and configure their ingestion pipeline
• Handle the diversity of source formats: CSV files with inconsistent headers, Excel workbooks with multiple sheets, database dumps, EMR exports with vendor-specific quirks, and FHIR API responses
• Work with healthcare org contacts to understand their data: what each column represents, which fields are populated vs. sparse, what their date formats and currency conventions are
• Document each org's data profile: source system, delivery method, column mappings, known data quality issues, and any org-specific transformation rules

Data Validation & Quality Assurance

• Run the automated validation suite on every new data load — verify schema compliance, data type correctness, value ranges, and analytics compatibility
• Investigate and resolve validation failures: missing required columns, unexpected data types, out-of-range values, duplicate records, encoding issues
• Extend validation rules when new edge cases surface (e.g., a hospital that uses a different claim status taxonomy, a pharmacy with drug names in a non-standard format)
• Perform manual spot-checks on analytics output after loading new org data — verify that KPI calculations, trend charts, and LLM-generated analytics produce sensible results for the new dataset

Pipeline Operations & Monitoring

• Monitor ingestion pipeline health across all active organisations: check for failed loads, state data, schema drift, and data quality alerts
• Troubleshoot pipeline failures: parse errors, connection timeouts, rate limits on FHIR APIs, schema changes from source systems
• Run incremental data loads when orgs provide updated data (new months, corrections, backfills)
• Maintain pipeline configuration: connection credentials (rotated securely), schedule configurations, retry policies

Data Transformation & Cleaning

• Write and maintain transformation scripts that handle org-specific data quirks: column renaming, date format conversion, currency normalization, categorical value standardisation
• Handle the column normalization that Helda requires
• Build and maintain mapping tables for categorical standardization
• Clean and deduplicate records where source data has quality issues, documenting all transformations applied