Skip to content 
Meet the dbt Fusion Engine: a game-changer in data transformation
In January 2025, dbt Labs announced the acquisition of SDF Labs, a company known for its deep expertise in advanced SQL parsing and execution. This was a major leap forward for the analytics engineering community. This technology now powers the new dbt Fusion engine, which represents the most substantial upgrade to dbt Core’s internals since […]
Automate Invoice Classification with GenAI: A Multi-Stage Approach on Snowflake
Dealing with mountains of invoices is a familiar headache for many finance and HR departments. Just imagine manually classifying thousands of those invoices each month against a complex list of over 300 categories, ranging from office supplies and training to travel and specific service maintenance contracts. This was a challenge that one of our clients […]
DSPy Prompt Automation, Evaluation, and Optimisation: Building a Financial Adviser
DSPy stands for Declarative Self-improving Python. Moving away from brittle prompts, you can now write compositional Python code and use DSPy to teach your Large Language Model (LLM) to deliver high-quality output. Approaches like Chain of Thought (CoT) and Reasoning and Acting (ReAct) are powerful, but they still rely on carefully constructed examples and prompts. […]
MCP vs Workflow Automation vs LLM-specific function calling for Enterprise
Model Context Protocol (MCP) is an open protocol that standardises how applications provide context to LLMs. You can think of MCP like a USB-C port for AI applications. With the success of GenAI in the consumer space, there is no shortage of tools and adapters to connect to AI applications. Generally, there are three types […]
From R Scripts to Azure Magic: How to Hack Azure Data Factory to Run R
Picture this: You have got a stack of R scripts — perfect little data-crunching, web-scraping gems — and a client who is adamant they need to run inside Azure Data Factory. One problem: ADF does not speak R. What do you do? In this article, I will let you in on how to run R […]
Performance Impact of PostgreSQL Triggers
First things first, what are PostgreSQL triggers? PostgreSQL triggers are special functions that automatically execute in response to specific database events, such as inserts, updates, or deletes, on a table (or on materialised views with some limitations). Think of them as built-in listeners: when something changes in your data, a trigger can step in and […]
Reliable GenAI for Enterprise: Scalable, Interoperable & Programmable
Generative AI has made tremendous strides in consumer-facing applications — we have all witnessed the rapid adoption of ChatGPT, Claude, and others. At enterprise level however, productionising GenAI remains a major challenge. Simple chatbot solutions quickly become outdated compared to cutting-edge features, such as voice mode in ChatGPT, and deeply integrated use cases often prove […]
Real-Time Dashboard with Tinybird
Tinybird has achieved substantial increase in analytical power over ksqlDB and comparable products, due to its introduction of ClickHouse database. At Infinite Lambda, we evaluated Tinybird over some complex joins operation over 200 million of rows, and found the latency to be well within 30 seconds. Tinybird offers data lineage similar to Snowflake Dynamic Tables/ […]
Reduce Latency with Snowflake Dynamic Tables and Minimal Rewrite of Existing dbt Models
For enterprises which already have Snowflake as the cloud data warehouse, the Snowflake dynamic tables, coupled with Native Snowflake Connector for PostgreSQL, could substantially reduce latency with minimal rewrite of existing models. In particular, dbt 1.9 has broadened the support for Snowflake dynamic tables, enabling refresh mode selection. We find the end-to-end latency of a […]
Real-Time Analytics: The New Approach You Need
Being able to analyse information in real-time is no longer a luxury but a necessity for enterprises that seek to maintain a competitive edge. From monitoring live dashboards to enabling rapid AI model retraining, demand for up-to-the-minute insights has surged. Meeting the technological expectations that enable real-time analytics requires a delicate balance between latency and […]
A/B testing an ML-powered learning-to-rank algorithm
![A/B testing an ML-powered learn-to-rank algorithm [part 2]](https://infinitelambda.com/wp-content/uploads/2025/01/A-B-testing-an-ML-powered-learn-to-rank-algorithm-1024x538.png){kind=icon}
Personalisation is one of the most impactful applications of machine learning. In this article, we will explore the development and A/B testing of the ML-powered learn-to-rank (LTR) algorithm we helped one of our clients to build. The solution was created for Secret Escapes, a leading members-only online travel deals company with millions of users worldwide. […]
Building an ML-powered Learning-to-Rank Algorithm
![Building an ML-powered learn-to-rank algorithm [part 1]](https://infinitelambda.com/wp-content/uploads/2025/01/Building-an-ML-powered-learn-to-rank-algorithm-blog-post-1-1024x538.png){kind=icon}
One of the most powerful uses of machine learning technology is personalisation. In this article, we will go into detail about building an ML-powered Learning-to-Rank (LTR) algorithm for a client in the travel industry. Secret Escapes is one of the world’s leading members-only online travel brands, offering hand-picked offers to millions of users worldwide. They […]