WhatsApp voice-note order automation, the macOS path that skips Whisper and the Business API

No. The MCP server's read tool walks the macOS accessibility tree and only emits text nodes whose descriptions start with 'message, ' or 'Your message, '. An untranscribed voice bubble exposes a duration and a play button to AX, not a transcript, and those get dropped by the uiSkipKeywords filter at main.swift line 535. The transcription happens inside WhatsApp itself on-device; the MCP just reads the result. If you want a sidecar Whisper pipeline you can absolutely build one, but for a single-Mac order flow it is wasted work.

On iOS each voice message has a Transcribe button that runs on the recipient's device on demand. On macOS the desktop client surfaces the same expandable transcript on bubbles in the conversation. The setting lives at Settings > Chats > Voice message transcripts and you can also pick the transcript language. Transcription is on-device, so end-to-end encryption is preserved. If both sides are using clients that do not support it, the audio bubble stays opaque to the AX tree and your MCP read will see only the typed messages around it.

On iPhone WhatsApp transcription supports English, Spanish, French, German, Italian, Japanese, Korean, Portuguese, Russian, Turkish, Chinese, Arabic, Hebrew, Swedish, Thai and several more. On Android the list is shorter: English, Portuguese, Spanish, Russian. The macOS desktop client follows the iPhone list because it pairs with the user's iPhone account. If your customers send orders in a language not on either list, you fall back to the sidecar pattern (download the audio out-of-band, run it through a multilingual model).

Yes if the bubble has already been transcribed in the WhatsApp UI, no if it has not. WhatsApp does not pre-transcribe every voice note in the background; it transcribes when the user (or your test account) taps Transcribe. For a real ordering pipeline you have two practical options. One: set a sticky habit in your operations Mac of tapping every new voice bubble so the transcript is materialized before the agent polls. Two: script that tap. The MCP does not expose a tap-to-transcribe tool today, but a small additional handler that finds the Transcribe button by its AX label and clicks it would be straightforward, and the existing click primitives in main.swift are reused unchanged.

Twilio plus Whisper is the right answer if you need a hosted, multi-tenant pipeline that serves thousands of voice-note orders per day across many merchants. You get an audio file by webhook, you transcribe with a model you control, you store everything in a database you own. For a single owner-operator who already runs WhatsApp on a Mac and gets 10 to 200 voice notes a day, that whole stack is overhead. The desktop-MCP path replaces it with: enable transcription, install one npm package, give your LLM read access. Net cost for the inbound voice-note layer goes to zero.

The Business Cloud API supports inbound media including voice notes, but it does not transcribe them. You get a media ID, you fetch the binary, and you transcribe it yourself. You also lose the customer's existing chat thread because the customer now has to message your verified Business number, not the personal one they have been ordering from for two years. For an existing book of regulars who already voice-message you on WhatsApp, switching them to a Business Cloud number is the most expensive move of the whole project. Keep the chat where it is, run an MCP server beside it.

Not today through this MCP. The send tool sends plain text and verifies the bubble appeared. The MCP's llms.txt at line 118 is explicit: 'Text messages only. The send tool handles text. It does not support sending images, files, voice messages, or other media.' For an order pipeline this is fine because a typed confirmation that quotes the order back is clearer than a voice reply anyway. If your customers strongly prefer voice replies, generate audio out-of-band (ElevenLabs, Apple Speech), drop the file into the WhatsApp paste buffer, and trigger a paste-and-send through a sibling tool. That extension would belong in the MCP, not above it.

macOS only, single WhatsApp account, single Mac, WhatsApp Desktop must be running and visible to AX. WhatsApp's transcription is on-device, which is great for privacy and bad for latency on older Macs and phones with smaller models. You are still subject to whatever rate-limiting WhatsApp applies to a normal personal account. And the LLM is reading a transcript, not the original audio, so if the transcript is wrong about a quantity (two becomes too, four becomes for) the order will be wrong unless your validator catches it. Add a confirmation step that quotes the parsed order back to the customer, and treat any reply that is not 'yes' as a re-parse trigger.