Recent content on Strathweb. A free flowing tech monologue.
Back in December I introduced the MLX Integration Library for Agent Framework, a small library that lets you plug local MLX models into Agent Framework applications as a regular chat client, sitting right next to your cloud-backed agents. Agent Framework has been moving quickly since then, and rather than letting the library drift behind, I spent some time catching up and, in the process, added…
In part one of this little series I introduced TasmanianDevil , a standalone, offline PII detection and de-identification engine for .NET. We saw it on its own - detecting and validating PII, anonymizing it with a range of operators, the reversible encrypt/decrypt round-trip, structured JSON and CSV redaction, and the optional multilingual NER add-on. That engine is useful anywhere, but the place…
A few months ago I introduced AgentGuard , a library for declarative guardrails and safety controls for .NET AI agents. One of the rules it shipped with from day one was PII redaction, but back then it was a fairly basic, regex-only affair - good enough to scrub an email address or a credit card number, but not much more. Since then I have rebuilt that part of the library from the ground up into a…
Series overview 🔗 [Part 1: The Basics][7] Part 2 (this part): Using the tools directly via the SDK 🔗[1]:
This post continues a series on hybrid architectures that combine local Small Language Models (SLMs) with cloud Large Language Models (LLMs). We have already looked at the Minions pattern , which offloads bulk text extraction to a local model to reduce cloud API costs, and the SLM-default, LLM-fallback pattern , where a local model handles the majority of queries and only escalates to the cloud…
Some years (wow, time flies!) ago I wrote a quantum computing book titled “Introduction to Quantum Computing with Q# and QDK” (I guess it’s difficult to miss too, since its cover still sits in the sidebar of this blog…). Even though a ton of things have changed in QDK since than, I have been maintaining the source code throughout the years - most importantly, porting…
As AI agents become more common in .NET applications, the question of how to keep them safe and well-behaved keeps coming up. Prompt injection, PII leakage, topic drift, tool call abuse - these are all problems that every team building with agents ends up having to deal with, often by hand-rolling ad-hoc checks. Python developers have had libraries like NeMo Guardrails and Guardrails AI to help…
Over the last few years, I have dedicated a lot of space on this blog to the topic of Post-Quantum Cryptography (PQC). Today, we will peek into the TLS 1.3 handshake. While hybrid mode TLS is the pragmatic choice for today’s internet, understanding how to construct a fully quantum-safe connection is critical for preparing for the deprecation of classical algorithms . In this post, we are…
Recently, I dedicated quite a lot of room on this blog to the topic of running Phi locally. This time, I want to focus on a different aspect of adopting small language models like Phi - fine-tuning them. I already covered local fine-tuning in the past , so today we are going to do this with Azure Machine Learning (Azure ML) . Azure ML is a comprehensive cloud service for accelerating and managing…
Following the initial introduction of (Maybe) LibOQS.NET a few months ago, the post-quantum cryptography landscape has continued to move rapidly. To keep pace with these developments, the library has undergone significant updates to provide broader algorithm support and deeper integration with the underlying native tooling. Today marks the release of version 0.3.0 , a major milestone that brings…
In the world of quantum computing, generating true randomness is one of the most fundamental applications. But how do we know that a sequence of numbers is truly random and generated by a quantum process, rather than by a classical simulation or a pre-determined list? In this post, we will explore a protocol to generate high-quality random numbers using a quantum computer, based on the recent…
If you visited this blog before, chances are you are familiar with Q# Bridge , a library that I have been working on for quite a while, that allows you to run Q# quantum simulations and access a number of Q# compiler/QDK features from multiple popular high-level languages such as C#, Swift, Python and Kotlin. Today, I would like to shortly talk about a brand new feature in the library - the…
I’ve recently been working on setting up a bunch of Agent Framework samples, which would showcase the cooperation between cloud agents (backed by LLMs in the cloud) and local agents (running on your own machine). Since I primarily work on a Mac, the natural choice for me was to use MLX as the local model runner, which required a bit of bootstrapping - and felt quite tedious. So, the natural…
When building AI workflows, we often face a choice: do we use a massive, expensive cloud model for everything (to ensure best reasoning capabilities), or do we cut costs with a smaller local model (and risk hallucinations)? In this post, we’ll explore a “best of both worlds” architecture, as described in the recent survey “Small Language Models for Agentic Systems”…
Last week, we released version 2.0 of dotnet-script . The latest release introduces support for .NET 10.0 and C# 14 and is available, as usually, through Github releases and on Nuget. You will need to have at least the .NET SDK 10.0.100 installed. It is the first major release of dotnet-script since version 1.0, which was released back in 2020. At the same time, the breaking changes are minimal,…
In this post, we’ll explore a novel approach to optimizing AI workflows by strategically combining large language models (LLMs) with small language models (SLMs) using the “Minions pattern.” This technique, described in the research paper “Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models” by Narayan et al., addresses one of the most…
Over recent years I have been involved in the post-quantum cryptography community, especially from the .NET angle - trying to streamline integration of PQC into .NET space and raise the awareness of developers via various projects, samples and articles . In this spirit, I would like to announce today a library called (Maybe) LibOQS.NET , which is a thin wrapper around liboqs , a C library…
Following up on my recent posts about ML-KEM and ML-DSA post-quantum cryptography in .NET using BouncyCastle.NET , I wanted to share an interesting development on the Windows side. Microsoft has recently announced post-quantum cryptography support in Windows through their Cryptography API: Next Generation (CNG) libraries. This development provides an alternative to third-party libraries for…
In this post we will explore a novel approach to Retrieval-Augmented Generation (RAG) called HyPE (Hypothetical Prompt Embeddings) , which I came across in a preprint paper recently. This technique tries to address one of the fundamental challenges in RAG systems: the semantic mismatch between user queries and document content. If you’ve ever built a RAG system, you’ve probably felt…
In the previous part of this series, we explored how to attach OpenAPI-based tools to a Semantic Kernel AI agent. In this part, we will look at another SDK for building AI Agents, Azure AI Foundry SDK, to create an agent that can also interact with OpenAPI-based tools.
Today we will kick off a short series on building AI agents which have access to OpenAPI tools. In this first part, we will focus on the Semantic Kernel, and in the second part, we will look at Azure AI Foundry.
There is an old joke that one day a cat jumped on a keyboard and that’s how Perl was invented. If you ever worked with array updates in Q#, you have certainly felt the pain of the syntax. At least for me, it was always a source of confusion and frustration. It was one of those things, like regular expressions, that I needed to look up every time - and then I would immediately forget it…
Last week at Build, Microsoft released a public preview of Foundry Local - a new tool for running AI models locally on your machine, with a focus on privacy and security. It is a spiritual sibling to Azure AI Foundry , which is a managed Azure cloud service for building and running AI applications. However, using Foundry Local is independent of Azure, and it can run models locally without any…
Last year , Microsoft announced the Copilot Plus PC, a new class of devices that are designed to run AI workloads locally. The flagship device of the line is of course the Surface Pro 11 , which is powered by the Qualcomm Snapdragon X Elite ARM processor. Unfortunately, since the launch, the AI capabilities have been more than underwhelming, as few applications and workloads are able to take…
If you have tried to use the OpenAI o-series reasoning models, such as o1 or o3 , with PromptFlow recently, you certainly ran into a nasty surprise. While PromptFlow supports a wide range of models and providers, the o-series models are not among them. This is of course quite a shame, especially if you’d like to benchmark or evaluate your flows against those models. In this short post, we…
Quokka (or “Quokka Puck”) is a brand new quantum educational device from Chris Ferrie and Simon Devitt , originally funded by a Kickstarter campaign . It is a beautifully designed, standalone, plug and play, 30-qubit quantum computer emulator. Sure you can simulate quantum programs on your laptop, but Quokka is a dedicated device that is designed to make quantum computing education…
As I previously blogged a few times, I have been working on the Strathweb Phi Engine , a cross-platform library for running Phi model inference via a simple, high-level API, from a number of high-level languages: C#, Swift, Kotlin and Python. This of course includes the capability of running Phi models on iOS devices, and the sample repo contains a demo SwiftUI application that demonstrates how to…
Last summer, I launched Strathweb Phi Engine — a cross-platform library for running Phi model inference via a simple, high-level API, from a number of high-level languages: C#, Swift, Kotlin and Python. Today I am happy to announce support for Phi-4, the latest model in the Phi family, which Microsoft AI released in December 2024.
Some time ago, I wrote a post about post-quantum cryptography in .NET , where I introduced the concept of post-quantum cryptography and discussed the early BouncyCastle.NET implementation of Kyber and Dilithium. Today I would like to revisit this post, as both of these algorithms have been officially standardized as ML-KEM and ML-DSA.
Recently, I dedicated quite a lot of room on this blog to the topic of running Phi locally with the Strathweb Phi Engine . This time, I want to focus on a different aspect of adopting small language models like Phi - fine-tuning them. We are going to do this with Apple’s MLX library, which offers excellent performance for ML-related tasks on Apple Silicon. We are going to do LoRA fine tuning…
Local AI inference has become increasingly important for developers seeking to build robust, privacy-preserving applications. In this deep dive, I’ll show you how to leverage Strathweb Phi Engine multi-platform library to run Microsoft’s Phi-family models directly in your .NET applications, exploring both basic integration patterns and advanced features that make Phi inference more…
For a few years in a row now, around this time of the year, I have been writing a festive Q# quantum computing post. This year I would like to keep the tradition going and explore another fun topic . Ever wondered what would happen if we let quantum mechanics decorate a 🎄 Christmas tree ? Let’s explore a quantum program - in both my favorite quantum programming language, Q#, as well as in…
In the summer of 2024, I announced the Q# Bridge library , which allows you to run Q# simulations from many popular high-level languages - C#, Swift, Python and Kotlin. Today, I would like to write about a brand new feature in the library, an ability to generate OpenQASM 2.0 code from Q# source. This is a feature that Q# toolchain does not natively supports, and it adds to the value proposition of…
In this post, we will explore the flexibility behind Azure AI Inference, a new library from Azure, which allows us to run inference against a wide range of AI model deployments - both in Azure and, as we will see in this notebook, in other places as well. It is available for Python and for .NET - in this post, we will focus on the Python version.
This summer, I announced the Strathweb Phi Engine — a cross-platform library for running Phi inference anywhere. Up until now, the library only supported models in the quantized GGUF format. Today, I’m excited to share that the library now also supports the Safe Tensor model format. This enhancement significantly expands the scope of use cases and interoperability for the Strathweb Phi…
GPT-4o-mini is the small, cost-effective version of the GPT-4o model. It is a great default choice for developers who want a very capable and fast model, but don’t need the full power of the GPT-4o model. However, there are some important things to keep in mind when using GPT-4o-mini, especially when it comes to pricing - some of which is rather contradictory!
On October 1st, OpenAI and Microsoft (Azure OpenAI) announced the availability of the GPT-4o Realtime API for speech and audio. It is a new, innovative way of interacting with the GPT-4o model family, the provides a “speech in, speech out” conversational interface. Contrary to traditional text-based APIs, the Realtime API allows sending the audio input directly to the model, and…
I recently announced Strathweb Phi Engine , a cross-platform library/toolset for conveniently running Phi-3 (almost) anywhere. Today I would like to show how to integrate a local Phi-3 model, orchestrated by Strathweb Phi Engine, into an agentic workflow built with AutoGen .
I previously blogged about post-quantum cryptography on this blog a few times . Among other things, I released a set of helper libraries for working with Dilithium in .NET and Duende Identity Server, as well as shared some general samples on post-quantum cryptography in .NET. Earlier this month, in a big milestone, NIST released the first 3 finalized Post-Quantum encryption standards. I thought it…
Earlier this week, I released version 0.2.0 of my post-quantum cryptography helper library .NET, Strathweb.Dilithium , which introduces a new feature - automatic key management support in Duende Identity Server. This feature plugs into the automatic key management capabilities of Duende Identity Server, and allows you to automatically generate and manage Dilithium keys for token signing purposes,…
I recently wrote a blog post about using Rust to run Phi-3 model on iOS. The post received an overwhelmingly positive response, and I got a lot of questions about running Phi-3 using similar approach on other platforms, such as Android, Windows, macOS or Linux. Today, I’m excited to announce the project I have been working on recently - Strathweb Phi Engine, a cross-platform library for…
Many years ago I wrote a book about ASP.NET Web API. One of the chapters in that book was dedicated to supporting push communication between the server and the client, and one of the covered techniques was the niche technology called Server-Sent Events ( SSE ). At the time, SSE was not widely supported by browsers, however, it was super simple and effective way to push data from the server to the…
Over the past year, Q# and the QDK, have undergone a massive transformation , with the entire toolchain moving to Rust - which resulted in a significant performance improvement, better portability of the toolchain and the ability to run Q# on a wide range of platforms. This was especially striking compared to the 0.x versions of the QDK, which was coupled to the .NET SDK. Today I would like to…
Last month, Microsoft released the exciting new minimal AI model, Phi-3 mini. It’s a 3.8B model that can outperform many other larger models, while still being small enough to run on a phone. In this post, we’ll explore how to run the Phi-3 model inside a SwiftUI iOS application using the minimalist ML framework for Rust, called candle , and built by the nice folks at HuggingFace.
Last time around , we discussed how Large Language Models can select the appropriate tool and its required parameters out of freely flowing conversation text. We also introduced the formal concept of those tools, which are structurally described using an OpenAPI schema . In this part 2 of the series, we are going to build two different .NET command line assistant applications, both taking…
One of the fantastic capabilities of the Large Language Models is their ability to choose (based on a predefined set of tool definitions) the appropriate tool and its required parameters out of freely flowing conversation text. With that, they can act as facilitators of workflow orchestration, where they would instruct applications to invoke specific tools, with specific set of arguments. OpenAI…
In my recent posts , I’ve been exploring various facets of the Azure OpenAI Service , discussing how it can power up our applications with AI. Today, I’m taking a slightly different angle - I want to dive into how we can enhance our projects further by integrating Azure OpenAI Service with Azure AI Speech . Let’s explore what this integration means and how it could lead to…
For our Retrieval-Augmented-Generation (RAG) application, we setup AI Search in part 1 , however so far we only used it using the basic keyword search. In this part 4 of the series about bringing your own data to Azure OpenAI Service, we will go ahead and integrate vector search, as a more sophisticated way of performing the search across the Azure AI Search index within our RAG-pattern system. I…
It is a common scenario to use Azure DevOps to build, sign and release iOS applications. Most of the tasks related to that can be handled by the Xcode@5 task, which provides support for all kinds of build activities around Xcode workspaces, and which is a de-facto shortcut for invoking xcodebuild command line tool. The task is quite well documented, but it is not entirely obvious how to use it for…
As you may know from the announcement on this blog (or from the image in the sidebar…), I wrote a quantum computing book titled “Introduction to Quantum Computing with Q# and QDK” which was published by Springer in May 2022. The source code samples used in the book were written against version 0.21.2112180703 of the QDK and the Q# language, which had been released on 14th…