Data Infrastructure Transformation
Designed and re-implemented the data infrastructure and data model of a growing FinTech, reducing the data-related cloud resource spend by >75% whilst also improving other performance measures such as latency and time-to-ship new features.
The data infrastructure was moved from an expensive and inflexible platform as a service to a much more cost-efficient, scalable, and extensible data platform using open-source ELT and orchestration tools implemented in their existing AWS account.
Testing and deployment via CI/CD workflows were implemented, along with better practices around project/feature specific development environments, reducing the risk and effort in releasing improvements and features (especially those that impacted their external customers).
Working in an agile way ensured both the data platform could be assessed under real continuous usage conditions and improved as we went, and developer resource could be pulled in or reduced as required to speed up delivery whilst minimising development cost.
Predictive Maintenance Solution
Built and deployed an architecture to execute the ML product lifecycle for an embedded IoT based manufacturing solution, leveraging the training of models on powerful cloud resources whilst enabling the gathering of data and deployment of models to thousands of embedded devices for real-time local inference.
Alongside this, data collection was enhanced through implementation of a custom labelling tool application available to data scientists and shop floor workers alike to both enable the easy labelling of data in real-time and the rapid review of model-predicted labels.
Monolith to Microservices Migration
Designed and executed a data migration to support the back-end team in moving their whole financial transaction system from a monolith-based architecture to a micro-services architecture. This required significant coordination and collaboration with various client teams to understand the ins and outs of both the old and the new system, with all edge cases covered.
Tight deadlines were enforced to ensure the new system was in place and processing all new incoming transactions before the new year in order to meet investors' expectations and secure further funding. These deadlines were successfully met by all involved, allowing the company to receive the funding to continue on its trajectory of scaling its customer base and service features.