Upgrading Your Local Gemma Setup

In Part 1, I set up Google’s Gem­ma 4 run­ning local­ly on macOS with Olla­ma and cre­at­ed a sim­ple gemma shell func­tion. It worked, but it was bare-bones — no auto-start, no mark­down ren­der­ing, and the inter­ac­tive mode was just a pass-through to ollama run.

Here’s how I turned that basic short­cut into some­thing that feels like a pol­ished chat inter­face, all with­in a sin­gle shell func­tion.

The Problem with Plain Text

Gem­ma’s respons­es are mark­down — head­ers, code blocks, tables, lists. But in a raw ter­mi­nal, a mark­down table looks like this:

| Column A | Column B |
|---|---|
| value | value |

Not exact­ly easy to scan. I want­ed ren­dered tables with prop­er bor­ders, high­light­ed code blocks, and head­ers with visu­al weight. Enter Glow.

Pretty-Printing with Glow

Glow is a ter­mi­nal mark­down ren­der­er from Charm. Install it with Home­brew:

brew install glow

The sim­plest upgrade is pip­ing one-off prompts through Glow:

gemma() {
    # ... existing function ...
    if [[ $# -gt 0 ]] && command -v glow > /dev/null 2>&1; then
        ollama run gemma4:26b "$@" | sed $'s/\033\[[0-9;]*[a-zA-Z]//g' | glow
    else
        ollama run gemma4:26b "$@"
    fi
}

Now gemma "Explain the difference between concurrency and parallelism" gives a nice­ly for­mat­ted response with ren­dered head­ers, high­light­ed code blocks, and prop­er­ly drawn tables — right in the ter­mi­nal.

The command -v glow check means if Glow isn’t installed, every­thing still works nor­mal­ly. It’s a pure upgrade with no down­side.

One gotcha: ollama run out­puts ANSI escape codes for ter­mi­nal cur­sor con­trol. These are harm­less in a nor­mal ter­mi­nal, but Glow treats them as lit­er­al text — you’ll see arti­facts like [2D[K scat­tered through the out­put. The sed com­mand strips those escape sequences before Glow sees them.

Auto-Starting Ollama

One annoy­ance with the basic set­up: if you for­get to start the Olla­ma ser­vice, the com­mand just fails. Easy fix — check whether the serv­er is reach­able and start it auto­mat­i­cal­ly:

if ! curl -sf http://localhost:11434/ > /dev/null 2>&1; then
    echo "Starting Ollama service..."
    brew services start ollama
    while ! curl -sf http://localhost:11434/ > /dev/null 2>&1; do
        sleep 1
    done
fi

This pings the Olla­ma API end­point on localhost:11434. If there’s no response, it runs brew services start ollama and waits until the serv­er is ready before pro­ceed­ing. If Olla­ma is already run­ning, it skips straight to the mod­el with no delay.

Interactive Mode: The Hard Part

The one-off pip­ing trick does­n’t work for inter­ac­tive con­ver­sa­tions — pip­ing std­out through Glow breaks the inter­ac­tive input. So I took it a step fur­ther and built a cus­tom inter­ac­tive mode using the Olla­ma API direct­ly.

The goals:

1. Con­ver­sa­tion his­to­ry with­in a ses­sion — once I start a chat, the mod­el should remem­ber every­thing I’ve said in that chat until I exit. (No per­sis­tence across ses­sions — each new gemma invo­ca­tion starts fresh.)
2. Vis­i­ble progress feed­back — a spin­ner so I know the mod­el is work­ing, not hung, because it can take a bit of time for hard­er requests
3. Mark­down ren­der­ing — ren­der the final response through Glow
4. Grace­ful inter­rupts — Ctrl‑C should exit clean­ly, not leave orphaned process­es

Conversation History with the Chat API

Instead of shelling out to ollama run, the func­tion calls the /api/chat end­point with curl. This end­point accepts a messages array, which means we can main­tain the full con­ver­sa­tion his­to­ry for the dura­tion of the chat. I store the mes­sages as JSON in a temp file cre­at­ed with mktemp when the ses­sion starts, append each new exchange with jq, and the cleanup func­tion deletes that file when you exit — so his­to­ry lives exact­ly as long as the inter­ac­tive ses­sion does:

# Add user message to history
jq --arg p "$prompt" \
    '. + [{"role": "user", "content": $p}]' "$tmpfile" > "$tmpfile.tmp" \
    && mv "$tmpfile.tmp" "$tmpfile"
# Send full history to the API with stream:false
curl -sS http://localhost:11434/api/chat -d "$(jq -n \
    --argjson msgs "$(cat "$tmpfile")" \
    '{"model": "gemma4:26b", "messages": $msgs, "stream": false}')"

You’ll need jq for the JSON han­dling — it comes pre-installed on recent macOS ver­sions, or you can grab it with brew install jq.

A ​“Thinking…” Spinner While You Wait

A 26B-para­me­ter mod­el on a lap­top takes sev­er­al sec­onds per response, and dur­ing that time the ter­mi­nal needs to look alive. (In oth­er words, it’s slow­er than you may expect if you’ve used Claude, Gem­i­ni, or Chat­G­PT.) The trick is to send "stream": false, run curl in the back­ground, and ani­mate a spin­ner in the fore­ground until the response comes back — then ren­der the whole thing through Glow exact­ly once. The same pat­tern is reused in one-shot mode, so both code paths feel iden­ti­cal.

response_file=$(mktemp)
local _payload=$(jq -n --argjson msgs "$(cat "$tmpfile")" \
    '{"model": "gemma4:26b", "messages": $msgs, "stream": false}')
curl -sS http://localhost:11434/api/chat -d "$_payload" > "$response_file" 2>&1 &
local _curl_pid=$!
local _spinner='|/-\' _i=0
printf '\033[?25l'
while kill -0 "$_curl_pid" 2>/dev/null; do
    printf '\r\033[2;90mThinking... %s\033[0m' "${_spinner:_i++%${#_spinner}:1}"
    sleep 0.1
done
wait "$_curl_pid"
printf '\r\033[K\033[?25h'

Once the spin­ner clears, the response gets pulled out of the JSON with jq and ren­dered with Glow:

content=$(jq -r '.message.content // empty' "$response_file" 2>/dev/null)
if $has_glow; then
    printf '%s' "$content" | glow
else
    printf '%s\n' "$content"
fi

Why both­er with the HTTP API instead of just ollama run gemma4:26b "$prompt" | glow? ollama run buffers every­thing until the mod­el is done (no room for a spin­ner) and emits TUI escape codes that show up as garbage in Glow even when redi­rect­ed. The HTTP API gives you clean JSON and decou­pled con­trol over the UI.

Graceful Interrupt Handling

The trick­i­est part was get­ting Ctrl‑C to work prop­er­ly. A naïve trap ... EXIT in a shell func­tion applies to the entire shell ses­sion, not just the func­tion. The fix is a cleanup func­tion that resets the trap after run­ning:

_gemma_cleanup() {
    printf '\033[0m'
    rm -f "$tmpfile" "$tmpfile.tmp" "$response_file"
    trap - INT TERM
}
trap '_gemma_cleanup; return' INT TERM

This ensures Ctrl‑C kills the in-flight request, resets the ter­mi­nal col­or, cleans up temp files, and returns you to a work­ing prompt.

File References with @filename

In one-shot mode, you can pipe text files in with stan­dard redi­rec­tion (gemma "explain this" < file.sh). But in inter­ac­tive mode, stdin is already being used for your input. And pip­ing in a PDF? That sends raw bina­ry to the mod­el and it chokes.

So I added @filename syn­tax — pre­fix any file path with @ and the func­tion reads its con­tents into the prompt before send­ing it to the mod­el. It works in both one-shot mode:

gemma "explain this script @warm_cache.sh"

and inter­ac­tive mode:

>>> compare these two files @old_version.py @new_version.py
>>> what does @src/utils.js do?

The func­tion uses zsh’s ${(z)...} oper­a­tor to split the prompt into words, then checks each one for a lead­ing @. If the file exists, its con­tents are inject­ed inline, wrapped with mark­ers so the mod­el knows where the file starts and ends.

You need to use the full path to the file. If the file isn’t found, the func­tion tells you and stops before send­ing any­thing to the mod­el — in both one-shot and inter­ac­tive mode.

$ gemma "what does @sample.pdf say?"
File not found: sample.pdf
Use the full path, e.g. @/Users/john/Downloads/sample.pdf

PDF Support

For PDFs, @filename auto­mat­i­cal­ly extracts the text using pdftotext (from the poppler pack­age) instead of send­ing raw bina­ry:

brew install poppler

With pop­pler installed, you can ref­er­ence PDFs just like any oth­er file:

gemma "summarize @/Users/john/Downloads/contract.pdf"
Reading PDF: contract.pdf... (this may take a moment)

The func­tion detects the .pdf exten­sion, prints a sta­tus mes­sage so you know it’s work­ing, and runs pdftotext to con­vert it to plain text before inject­ing it into the prompt. PDFs can be large, and the mod­el needs time to process all that text — the sta­tus mes­sage pre­vents that ​“is it stuck?” feel­ing. If pop­pler isn’t installed, you get a warn­ing instead of garbage out­put.

The Final Function

The com­plete func­tion, avail­able below or in a gist, ties every­thing togeth­er.

gemma() {
    setopt local_options no_monitor no_notify
    if ! curl -sf http://localhost:11434/ > /dev/null 2>&1; then
        echo "Starting Ollama service..."
        brew services start ollama
        while ! curl -sf http://localhost:11434/ > /dev/null 2>&1; do
            sleep 1
        done
    fi
    local has_glow=false
    command -v glow > /dev/null 2>&1 && has_glow=true
    local _gemma_system='Respond in plain Markdown only. Do not use LaTeX or math delimiters ($...$, \(...\), \[...\]). Write units and symbols as Unicode directly (e.g. 0°C, 32°F, π, ², ³, ½) instead of \circ, \text{}, \frac, etc.'
    # One-shot mode: pass prompt directly
    if [[ $# -gt 0 ]]; then
        local _prompt="$*"
        local _w _fp _fc
        for _w in ${(z)_prompt}; do
            if [[ "$_w" == @* ]]; then
                _fp="${_w#@}"
                if [[ -f "$_fp" ]]; then
                    if [[ "$_fp" == *.pdf ]]; then
                        if command -v pdftotext > /dev/null 2>&1; then
                            echo "Reading PDF: ${_fp##*/}... (this may take a moment)"
                            _fc=$(pdftotext "$_fp" -)
                        else
                            echo "Warning: install poppler for PDF support (brew install poppler)"
                            return 1
                        fi
                    else
                        _fc=$(cat "$_fp")
                    fi
                    _prompt="${_prompt//$_w/$'\n\n--- Contents of '"$_fp"$' ---\n'"$_fc"$'\n--- End of '"$_fp"$' ---\n'}"
                else
                    echo "File not found: $_fp"
                    echo "Use the full path, e.g. @\$HOME/Downloads/$_fp"
                    return 1
                fi
            fi
        done
        # Use the HTTP API directly so we get clean text (no TUI escape codes
        # that `ollama run` emits even when redirected). Run curl in the
        # background and show a spinner until it finishes.
        local _out_file _payload
        _out_file=$(mktemp)
        _payload=$(jq -n --arg s "$_gemma_system" --arg p "$_prompt" \
            '{"model": "gemma4:26b", "messages": [{"role":"system","content":$s},{"role":"user","content":$p}], "stream": false}')
        curl -sS http://localhost:11434/api/chat -d "$_payload" > "$_out_file" 2>&1 &
        local _curl_pid=$!
        local _spinner='|/-\'
        local _i=0
        printf '\033[?25l'
        while kill -0 "$_curl_pid" 2>/dev/null; do
            printf '\r\033[2;90mThinking... %s\033[0m' "${_spinner:_i++%${#_spinner}:1}"
            sleep 0.1
        done
        wait "$_curl_pid"
        local _status=$?
        printf '\r\033[K\033[?25h'
        if [[ $_status -ne 0 ]]; then
            cat "$_out_file"
            rm -f "$_out_file"
            return $_status
        fi
        local _content
        _content=$(jq -r '.message.content // empty' "$_out_file" 2>/dev/null)
        if [[ -z "$_content" ]]; then
            echo "Error: no response from model"
            cat "$_out_file"
            rm -f "$_out_file"
            return 1
        fi
        if $has_glow; then
            printf '%s\n' "$_content" | glow
        else
            printf '%s\n' "$_content"
        fi
        rm -f "$_out_file"
        return
    fi
    # Interactive mode with spinner + glow render
    local tmpfile=$(mktemp)
    local response_file=""
    local prompt expanded content _w _fp _fc
    _gemma_cleanup() {
        printf '\033[0m'
        rm -f "$tmpfile" "$tmpfile.tmp" "$response_file"
        trap - INT TERM
    }
    trap '_gemma_cleanup; return' INT TERM
    jq -n --arg s "$_gemma_system" '[{"role":"system","content":$s}]' > "$tmpfile"
    echo "Chat with Gemma (type 'exit' or Ctrl-C to quit)"
    while true; do
        printf "\n>>> "
        read -r prompt || break
        [[ -z "$prompt" || "$prompt" == "exit" ]] && break
        # Expand @filename references to file contents
        expanded="$prompt"
        local _file_error=false
        for _w in ${(z)prompt}; do
            if [[ "$_w" == @* ]]; then
                _fp="${_w#@}"
                if [[ -f "$_fp" ]]; then
                    if [[ "$_fp" == *.pdf ]]; then
                        if command -v pdftotext > /dev/null 2>&1; then
                            echo "Reading PDF: ${_fp##*/}... (this may take a moment)"
                            _fc=$(pdftotext "$_fp" -)
                        else
                            echo "Warning: install poppler for PDF support (brew install poppler)"
                            _file_error=true
                            break
                        fi
                    else
                        _fc=$(cat "$_fp")
                    fi
                    expanded="${expanded//$_w/$'\n\n--- Contents of '"$_fp"$' ---\n'"$_fc"$'\n--- End of '"$_fp"$' ---\n'}"
                else
                    echo "File not found: $_fp (use the full path, e.g. @/Users/john/Downloads/$_fp)"
                    _file_error=true
                    break
                fi
            fi
        done
        if $_file_error; then
            continue
        fi
        jq --arg p "$expanded" '. + [{"role": "user", "content": $p}]' "$tmpfile" > "$tmpfile.tmp" \
            && mv "$tmpfile.tmp" "$tmpfile"
        # Non-streaming request with spinner (matches one-shot mode)
        response_file=$(mktemp)
        local _payload=$(jq -n --argjson msgs "$(cat "$tmpfile")" \
            '{"model": "gemma4:26b", "messages": $msgs, "stream": false}')
        curl -sS http://localhost:11434/api/chat -d "$_payload" > "$response_file" 2>&1 &
        local _curl_pid=$!
        local _spinner='|/-\' _i=0
        printf '\033[?25l'
        while kill -0 "$_curl_pid" 2>/dev/null; do
            printf '\r\033[2;90mThinking... %s\033[0m' "${_spinner:_i++%${#_spinner}:1}"
            sleep 0.1
        done
        wait "$_curl_pid"
        printf '\r\033[K\033[?25h'
        content=$(jq -r '.message.content // empty' "$response_file" 2>/dev/null)
        if [[ -z "$content" ]]; then
            echo "Error: no response from model"
            rm -f "$response_file"
            continue
        fi
        jq --arg c "$content" '. + [{"role": "assistant", "content": $c}]' "$tmpfile" > "$tmpfile.tmp" \
            && mv "$tmpfile.tmp" "$tmpfile"
        if $has_glow; then
            printf '%s' "$content" | glow
        else
            printf '%s\n' "$content"
        fi
        rm -f "$response_file"
    done
    _gemma_cleanup
}

Drop this into your .zshrc, run source ~/.zshrc, and you’ve got:

• gemma — inter­ac­tive chat that remem­bers every­thing you’ve said for the dura­tion of the ses­sion, with an ani­mat­ed spin­ner and Glow ren­der­ing
• gemma "your question" — one-shot prompt with the same spin­ner and Glow ren­der­ing
• @filename — include file con­tents in prompts, in both one-shot and inter­ac­tive mode
• PDF sup­port — @document.pdf auto­mat­i­cal­ly extracts text via pdftotext
• Auto-start of Olla­ma if the ser­vice isn’t run­ning
• Clean Ctrl‑C han­dling
• Grace­ful fall­back if Glow, jq, or pop­pler aren’t installed

What It Feels Like

The expe­ri­ence went from ​“use­ful but rough” to some­thing I find much more user-friend­ly. You type a ques­tion, see the spin­ner ani­mate while Gem­ma is think­ing, and then the Glow-ren­dered response appears all at once — tables with prop­er bor­ders, high­light­ed code blocks, and head­ers with visu­al weight. It feels less like talk­ing to a ter­mi­nal and more like a prop­er chat inter­face.

All of this runs entire­ly on your machine, with no data leav­ing your lap­top. Not bad for a shell func­tion.