Project Codename Summarizer Part 4 - Pushing The Technical Side

Back on the technical side. As a reminder, the prototype was directly calling the OpenAI APIs.

That's a big issue for shipping client apps for iOS and macOS.

Indeed, anybody can set up a proxy and inspect the network and easily steal your API key and make calls on your behalf.

You will end up with a 💸 huge bill for API calls you never made.

And if that happens and you revoke your key, old versions of your app will stop working, locking your users out.

Thankfully, there is a solution to that: AIProxy (not sponsored!). It's a platform that comes with a Swift client able to work with most AI providers.

The way it works is very powerful: you enter your endpoint and API key on the platform, but the platform doesn't store it. It stores just part of it.

And then gives you the second part (a partial key), as well as the proxy URL where you can call this service.

That means that the platform doesn't store your full key, and your app code doesn't contain your full key either. It's the combination of both that reconstructs the actual key at runtime.

And the library implements DeviceCheck in order to make sure that your credentials haven't been stolen and that your app binary is legit.

AIProxy also allows you to rate-limit and monitor your requests, just in case something goes wrong.

Since my layers were already decoupled, it was quite easy to switch from directly calling OpenAI to using the AIProxy client.

AIProxy, although open-source, is not widely known. At least not enough for an LLM to know it perfectly without needing a refresher.

So in order to avoid writing everything by hand, I had to find a way to teach it to my assistant of choice: Cursor with Gemini 2.5 Pro.

After a few trials and errors, I found the perfect workflow:

• I used https://uithub.com to turn a full repo into text that can be ingested by an LLM

• There, I increased the max tokens (Gemini 2.5 Pro has a context window of 1 million tokens, so we're safe)

  • With this request: https://uithub.com/lzell/AIProxySwift?accept=text%2Fhtml&maxTokens=1000000, I ended up with 324K tokens for the full implementation

• In Cursor, I went to Cursor Settings ➔ Features ➔ Docs ➔ Add a Doc, and added the above URL

• I gave it a name and added it.

I was then able to reference this doc (which is actually the full implementation of the library) in my prompts using @Docs.

That's much better than pasting it or referencing it from the web, since the indexing will probably optimize and transform it in a vector representation that might end up using fewer tokens in the context window.

Using this technique, it was way easier to ask Cursor to implement the AIProxy integration, it didn't hallucinate and it made a very good implementation, using best practices from the repository.

For transcription, I used the recently released (and expensive!) GPT-4o-transcribe API from OpenAI, that receives some audio and transcribes it into text.

It's more powerful and accurate than Whisper and benefits from the knowledge that 4o has.

There are a few small catches though:

• The input file is limited to 25 MB (you can easily reach that size for a long video).

• OpenAI recommends not cutting in the middle of a sentence. And now suddenly it's a bit more difficult.

So we already had two non-trivial use cases. And I added a third:

• I wanted users to be able to play back the added audio, and I wanted to replicate the UI of voice messages. So I need to generate waveforms for each audio file.

Thankfully, this is the exact kind of task where vibe coding shines:

• Limited scope

• Clear objectives (chunking, silence detection to avoid cutting mid sentence, and waveform generation)

• Self-contained utilities

Generating the code to do that was pretty much a one-shot with the right prompts, and I could even tweak the parameters afterwards.

That was a incredible gain of time once again.

This was basically the end of the technical phase. There was still some code to write, but I considered the app feature complete at this point. 🎉

However, there were two very important parts missing:

• UI/UX

• Final polish

See you next time for those!