H

Dr. Heiner Di Camillo

Data Engineer and AI Solution Architect

About me
hero image

Skills

I am happy to serve with my knowledge and experience

Techniques

DevOps

MLOps

AgentOps

Test Driven Development

Clean Code

Clean Architecture

Languages

Java, C#

Python

TypeScript

HTML, CSS, PHP

C, Fortran

Dart

I have extensive experience with these frameworks

Backend

Spring, FastApi, .NET, Firebase, NodeJS

Database

Postgres, MySQL, MongoDB

Realtime Data

Kafka, Doris, Druid, Firestore

Machine Learning

LangChain, LangGraph, Tensorflow, Keras, sklearn

Cloud & Deployment

Kubernetes, AWS, GCP, Azure, Grafana

CI/CD

GitHub, Gitlab, Jenkins

Frontend and SSR

React, Next.js, Flutter, React-Native

Releases

Apple App Store, Google Play


My Services

I create software for you and with you

Elevate Your Projects with Expert Architecture

  • Robust Architecture: I provide sustainable architecture decisions accompanied by automated architecture tests, facilitating continuous feature development.
  • Streamlined Business Logic: Embrace the power of Test Driven Development for managing intricate business logic effectively.
  • Confident Deployment: Benefit from a stable CI/CD environment that empowers you to make informed business decisions.

Benefit from my Unparalleled Background

  • Scientific foundation: I thoroughly analyze complex systems, yielding understandable and dependable solutions. I minimize regression risk for ultra-stable systems.
  • Data Science and Software Development: Seamlessly integrate data science and software development within your team. Make one field benefit from the other.
  • Agile Collaboration: I seamlessly collaborate with developers, product owners, and departments.

Profit from my Presence

  • Efficient Communication: Experience clear and relaxed communication alongside focused development skills.
  • Insightful Enhancements: Anticipating project nuances, I offer cost-effective insights for supplementary features that enrich your project’s value.
  • Human Sensitivity: Communication sometimes happens between the lines. With active listening and common sense, I seize those treasures.

References

I have helped customers with various endeavours

Artificial Intelligence

LLM-driven Agentic Content Creation

Web Publisher

+

-

  • Designed and implemented agentic publishing architecture
  • Evaluated agentic AI-frameworks for autonomous multi-agent collaboration

Stack: LangGraph, LangChain, AutoGen, CrewAI, Python, Fastapi, Kafka, React

Duration: 8 Months and ongoing

Driven by competition, publishers strive do reduce their costs for publishing content entities like articles and videos. This is especially important for low-risk content with its small profit margin.

The goal therefore was: Create an automated content creation and publishing pipeline for low-risk content, allowing a human-in-the-loop to supervise all steps if required.

Taking a movie-news portal as the pilot project, I designed an architecture that allows humans and AI-agents to act collaboratively on the same abstraction level: The Unified Editorial Interface, which combines maintaining structured data (a movie database) and embedding structured data into publication entities (articles).

I evaluated which agentic framework (LangGraph, Autogen, CrewAI) would fit best into the architecture, finally settling towards LangGraph due to its mature state machine modelling.

The deployed solution then consisted of event-driven half-autonomous AI-agents, consuming and publishing to Kafka Topics. I created a custom React-driven Material-UI to allow human supervision and intervention in the news-event driven content creation pipeline.

Realtime Anomaly Detection

Web Publisher

+

-

  • Designed and implemented realtime anomaly detection system
  • Integrated anomaly notifications within the client's data mesh architecture

Stack: Kafka, Druid, Spring, Java, Python, Kubernetes, Grafana, AWS

Duration: 9 Months

In the fast-paced news industry, quick reactions to real-time developments are crucial. Our client, a web publisher, operates within a complex publication structure. Having transitioned from traditional print to online publication, they manage a network of independently branded websites, fed by content from a central editorial office.

Editors require real-time insights into topics’ performance, to push high-performing topics and improve those that are declining. To support editorial decisions, I conceptualized a system to alert editors and relevant staff about anomalies in traffic patterns across different categories. Additionally, technical use cases were implemented to notify about anomalies in traffic sources.

Anomaly detection relied on real-time data from an Apache Druid instance. We employed algorithms that analyzed past data and compared them with forecast estimates. When an anomaly was identified, the report was streamed into a Kafka topic. Downstream services managed push notifications to end-users.

The solution seamlessly integrated into the client’s data mesh architecture. I followed an MVP approach: Product features were iteratively developed in collaboration with end-users. I designed the system for production readiness, incorporating monitoring and alerting through Grafana.

Reinforcement Learning For A Foosball Robot

Factory Automator

+

-

  • Applied Reinforcement Learning in Unity-Simulation
  • Integrated trained AI-agent with hardware

Stack: TensorFlow, Python, Unity, OpenCV, Docker

Duration: 15 months

For a robotic foosball table demo, human players faced off against a robotic contestant. Equipped with industrial-grade motors and actuators, the robot used a camera to monitor the field. My team and I enhanced the robot’s control software using reinforcement learning.

We precisely replicated the foosball table in a Unity simulation. The AI trained itself through continuous self-play in Unity, leveraging the SpinningUp reinforcement learning framework. Our self-developed hardware drivers bridged the gap between AI and real robot table. The AI’s dynamic gameplay even astonished us developers.

I maintained AI complexity with Test Driven Development, using test suites as feature documentation. This approach facilitated smooth onboarding for temporary team members on multiple occasions.

MLOps for Computer Vision in Cloud And Edge Devices

IoT Platform

+

-

  • Developed and deployed Machine Learning Pipelines
  • Trained, quantized, and accelerated Deep Learning Computer Vision models

Stack: Python, TensorFlow, Tensorflow Lite, Android, Docker, AWS

Duration: 12 months

A security infrastructure provider created an IoT platform for security cameras, allowing AI models for scene recognition to function as apps. I worked in an interdisciplinary team, researching and developing demo apps for common use cases.

Cloud-based model training and deployment (Python, TensorFlow, Terraform, Docker) laid the foundation. I contributed to the data-driven architecture, crafting scalable machine learning pipelines for inference and re-training feedback loops.

We honed models with quantization for mobile devices’ hardware acceleration (e.g. Qualcomm Hexagon). Quantized (MobileNet) and unquantized models, plus server-side models (e.g. ResNet), underwent performance comparisons.

Deployment occurred within Android-based cameras, acting as the app shell.

Classification Of Anonymized Movement Data

Security Provider

+

-

  • Developed machine learning pipelines for behavior classification
  • Employed clustering techniques on multivariate timeseries data

Stack: Python, Scikit-learn

Duration: 3 months

Analyzing anonymized personal movement data from supermarket cameras posed two challenges: data cleaning and behavior classification.

We calculated person movement similarity using dynamic time warping (DTW). For behavior clusters, we applied k-medoids and hierarchical agglomerative clustering on multivariate data. Using cluster centroids as classifier archetypes, our AI prototype accurately classified customer entering and exiting, validated against manual counts.

Our deliverable was a containerizable software suite assessing algorithmic strategy performance.

Assistant AI For A Packaging System

Supply Chain Manager

+

-

  • Analyzed and engineered requirements for assistant AI for user input suggestions
  • Developed machine learning model for multi-label prediction

Stack: Python, Scikit-learn, SQL

Duration: 6 months

To streamline industrial deliveries, users of the customer system grappled with a complex web-interface. Creating a packaging strategy required numerous repetitive manual steps due to years of feature accumulation that increased software power but sacrificed usability.

My team scrutinized requirements, delivering an AI prototype. We collaborated extensively with various departments, fostering enhanced communication within the customer organization.

Accessing sensitive production data for AI training required a secure pattern involving remote desktop connections and VPN. Our AI employed a random forest classifier with multi-label output. This algorithm learned packaging strategies from past deliveries, suggesting them to users.

The successful AI-POC spurred a production-ready order.

Fullstack and DevOps

Unified Storage Solution Evaluation

Web Publisher

+

-

  • Functional evaluation of Apache Doris as Data Lakehouse implementation
  • Load tests of Apache Doris with Kafka streaming and use cases of relational DBs

Stack: Apache Doris, MySQL, Kafka

Duration: 3 Months

The publishing platform of my customer was already event-driven in its core. But there were many subsystems from which master data was joined into the publishing stream.

My client’s goal was to implement prescriptive analytics into their processes, and to enable autonomous AI-agents to make use of the vast archives of article data and streaming statistics.

Apache Doris is a distributed analytics/big data Database which exposes a MySQL-compatible interface. This means that query-time joins between analytics-events and master data should be possible, e.g. in a star-schema.

I evaluated whether one Apache Doris Cluster could not only replace the Apache Druid analytics core system (and enhance it with joining and analytics capabilities as a data lakehouse), but also replace the grown landscape of various instances of MySQL, MongoDB, ElasticSearch and Redis.

I vigorously tested Doris’ ingestion capabilities via Kafka, use cases for relational data modelling and querying, and using Doris as a document database and cache.

By running load-tests of various scenarios close to production queries, I compiled a report that helped the customer in the executive decision towards Apache Doris as a “Unified Storage” building block of their platform architecture.

Track and Trace for a Logistics Platform

Logistics Provider

+

-

  • Decoupled transformation logic for representing shipments
  • Implemented legacy connector from logistics platform to modern tracking provider

Stack: OracleDB, PLSQL, XSLT, Java, REST

Duration: 12 months and ongoing

Relying on a proprietary legacy system, my client ships containers for international corporations throughout the whole world, using all available means of transport (road, rail and sea).

Their customers demand to be able to track their shipments in real time with a modern user interface.

The goal therefore was to implement change data capture for shipments in the legacy system. The legacy system also was strongly modified, using untested PLSQL-routines in their OracleDB. I created the data change increments using low-cost polling, then implemented a XSLT-based low-code transformation pipeline in another legacy connector platform of the client.

During the implementation, I was appointed to create an architectural proposal on the legacy systems and new customer facing systems could be decoupled to reach a faster growth velocity in onboarding new customers.

Interactive Geopolitical Publications

Web Publisher

+

-

  • Streamlined and expanded service architecture
  • Led frontend development and maintained geodata pipeline

Stack: React, NextJS, TypeScript, Kubernetes, AWS

Duration: 14 Months

My client’s team of data journalists aims to deliver engaging data experiences to readers, especially in mobile browsers. Their goals include providing real-time data on German regional elections and presenting an interactive geopolitical map of Ukraine.

In the dynamic news field, speed to production is paramount. To meet this demand, I merged two artifacts – a create-react-app frontend and a Node.js API—into a single NextJS application. This unified solution serves both React client code and essential data APIs, integrating seamlessly with the higher-level CMS of affiliated news sites.

I introduced a CMS-like feature, enabling editors to publish and embed new data visualizations without developer intervention. This approach reduces costs and organizational complexity.

Some data structures extended from the backend into the frontend, so I created decoupling points to enhance maintainability and extensibility. The streamlined architecture empowers junior developers to add new features with reduced complexity.

Productivity App for iOS and Android

My Own Solo-Startup

+

-

  • Multiplayer productivity app aiding users in decluttering their households
  • Managed full lifecycle of mobile app development, from feature design to app store release

Stack: Flutter, Firebase Backend, iOS, Android

Duration: 18 Months

My passion for only keeping essential things and living in an unburdened way led me create my own startup “Ramadama.”

“Let’s make some room!” - that’s what “Ramadama” means in Bavarian.

During COVID, when everyone was stuck in their room with all their stuff, I started with analyzing user needs, did some market research and found that helping people to declutter their life would be a great opportunity to create a platform.

I felt optimistic enough to create a functioning prototype of the app using my Flutter skills. I thought that doing user-research with a “living app” (meaning: a public beta) would give me quickly insights on how to design the core decluttering logic.

To make it easy for testers, I set up a complete app publication environment for both iOS and Android.

After a few months of iterations, I cancelled the project, because I alone by myself was unable to create an engaging and supportive UX for helping people declutter.

These are the essential points I learned from this endeavour:

  • Design the essential UX first before implementing anything.
  • Pivots in the UX of an app must be based on a flexible architecture. If you don’t have one, a rewrite might be quicker than adaption.
  • Don’t write software just for proving yourself that you can do it. You already know you can.
  • If your Design Sprints have inconclusive outcomes, the risk of creating a product without user engagement is real.
  • Publishing apps in app stores is great for marketing your app, but the overhead is very large for administration, building and testing.

For the sake of completeness, these were the most important features I chose to implement for the public beta:

Flutter Client:

  • Multiplayer mode
  • Localization (German, English, Italian)
  • Material Design
  • Reactive UI with BLoC and realtime updates
  • Custom theme with dark mode
  • Visual Corporate Identity

Firebase Backend:

  • NoSQL data base (Firestore)
  • Serverless Cloud Functions
  • Push-Notifications for user actions and engagements
  • Dynamic Links for intra-App-Actions
  • Scalability to millions of users

Connected Vehicle App

Car Manufacturer

+

-

  • Architected app's demo mode for millions of users
  • Led team and harmonized processes with nearshore colleagues (Romania, Germany)

Stack: Flutter, iOS, Android, TypeScript, .NET, NestJS, Docker, Terraform, REST, Jenkins

Duration: 19 months

Enhancing a Flutter app with a demo mode to showcase its benefits to potential users was our task. Requirements included ensuring features behaved “as if connected to a real vehicle”. However, fragmented feature development led to a lack of a complete development scope. We quickly overcame this by analyzing production UI and existing code.

With a newly-formed team split between Romania and our parent company, communication was vital. My nearshoring expertise facilitated seamless interaction. Building trust through clear architecture documentation bolstered our customer relationships.

As an architect, I restructured the app, allowing parallel development of demo features by individual developers. This loose coupling promoted scalability. Our successful demo mode launch led to us handling more core features. With team growth, I trained new developers, maintained high code quality, and architected new functionalities.

Tariff Ordering System

Telecommunications Provider

+

-

  • Incremental migration of extensive JavaScript frontend to React with TypeScript
  • Refactored microservice backend architecture and documentation

Stack: React, TypeScript, PHP, JavaScript, Jenkins

Duration: 9 months

Working collaboratively, we conducted an incremental migration of an extensive JavaScript frontend to React with TypeScript, alongside parallel development of new features. The existing code lacked tests and contained undocumented but vital business logic. We embraced Test Driven Development for our React codebase, enabling stable releases.

We assisted the customer in refining the front end’s logic, now backed by testing. This refinement extended to the legacy PHP backend. Through clear architecture and transparent inter-team communication, we relocated misplaced logic to intended services.

Our fully remote team included members from Germany, Hungary, and Egypt. Meanwhile, the customer organization transitioned to SAFe. As a Scrum Master, I guided the team through new processes, helping shape them and emphasizing the significance of CI/CD.

Quality Assurance Framework

Aircraft Manufacturer

+

-

  • Developed intuitive React-Redux frontend with Material UI
  • Designed API and built JavaEE backend

Stack: React, Redux, TypeScript, MaterialUI, Java, JavaEE, JBoss, Gerrit

Duration: 12 months

Operating within the customer’s SAFe framework, our multi-provider team embarked on standardizing a quality management process (9S), featuring a complex status model. To deliver an intuitive user experience, we harnessed Material-UI as the component library of choice.

Our outcome? A functionally-driven React frontend supported by a JavaEE backend. This backend, meticulously designed, retained full test coverage, facilitating agile responses to evolving requirements and streamlined logic patch releases. The beauty? Frontend remained free from business logic, while backend thrived. Our efforts helped navigate change and deliver seamless solutions.

Collapse all

Experience

Present

Data Engineer and AI Solution Architect

Freelancer / Self employed

I push my boundaries in the pursuit of creating value for customers.

2022
2022

Software Architect

Zielpuls GmbH – Part of Accenture

Here I found my love for Flutter and got involved in product design. A company with a great personal touch, yet powerful with international range.

2021
2021

Software Consultant

· Munich

TNG Technology Consulting GmbH

This engagement laid the foundation for my professional career in software engineering. An exceptionally profound and efficient organisation, always keen on learning and improving.

2018
2016

PhD in Physics/Meteorology

· Munich

Ludwig-Maximilians-University

I handled massive amounts of forecast and observation data with custom statistical software and high performance computing together with the German and Dutch Weather Service. My work led to a significant improvement of the operational weather forecast in Germany.

2013

Contact me

Let me know how I can support you or collaborate with you.


email icon linkedin icon