Do you need a chat app, an AI API or a local model?
Use a chat app to draft something you can review yourself. Connect an AI service when people need that help inside your app. Run a model locally when you need the work to happen on your own machine and have the hardware to do it.
Who needs the AI, you or the people using your app?
Picture the same booking question in three places. When you ask it in chat, you choose what to keep. When a visitor asks inside your app, the app needs to check the reply before the visitor sees it. On your own machine, you also supply the computer that runs the model.
| What you want | Where the work happens |
|---|---|
| Write a reply yourself with a little help | Paste a made-up request into a chat app, read the draft and choose what to keep. |
| Offer a draft inside your booking app | Your app sends a small request to an API, an agreed way for programs to talk. Your app checks the reply against the booking list before showing it. |
| Run the model on your own machine | A model runner loads the model files and works on that machine. Choose a local model and check whether any connected feature sends data elsewhere. |
Can you tell if a draft is useful before connecting anything?
Yes. Write down two available times for a made-up community room, Tuesday at 2 p.m. and Thursday at 11 a.m. The AI’s job is to suggest a reply in two sentences. The room’s booking list decides what is available.
You can do this on paper with examples you write yourself. Keep the availability, requests and expected replies in a file you control so you can repeat the check when you change the model. Use no customer details, connected tools, messages or real reservations.
- Give it a time you can checkUse “Tuesday afternoon” as the sample request. A useful draft offers Tuesday at 2 p.m. without promising a booking.
- Try a time that is unavailableUse “Friday please.” The reply should say Friday is unavailable and offer the two listed times. A plausible Friday time still fails the check.
- Leave the request incompleteUse “Can I book the room?” The reply should ask which of the two times the person wants.
- Decide what happens when it failsCross out any draft with an invented time or booking promise. A reply that cannot be checked can fall back to showing the two available times and asking the visitor to choose.
Compare tools
Scroll the table sideways to read every column.
| Tool | What do you sign up for?Chat, developer access or a local runner. | Where does the work happen?Which surface handles the prompt and reply. | What does your machine need?Whether it runs the model and what to check if it does. | What counts toward the limit?Which usage or bill you need to watch. | What should you track as it changes?The model or offering that can change the result. |
|---|---|---|---|---|---|
| ChatGPT | OpenAI’s chat app. Eligible Pro or Business subscriptions can claim credits in a linked OpenAI API organization. | Online. ChatGPT’s data controls and the API’s retention settings apply to their own surfaces. | Chat and the developer API use hosted models. | Check the app’s allowance or your API project usage, depending on how you connect. Keep a visitor’s allowance in your app. | Check the OpenAI API model notice separately from the models available in ChatGPT. |
| Claude | Claude’s chat app. Eligible Max, Ultra and Team plans can claim API credits in a linked Claude Console organization. | Online. Consumer chat and commercial/API policies have different training and retention terms. | The chat and Claude Platform API run remotely. | Check whether your workflow signs in with a Claude plan or uses a Console API key. They can draw from different allowances. | Confirm the model and availability in the actual chat or API surface before depending on it. |
| DeepSeek | Separate online chat, metered hosted API and downloadable releases from the same provider. | Direct hosted services have their own privacy terms. A downloaded release follows the runner and host you choose. | For downloadable releases, check the exact license, compatible runner, model files and available memory. | The direct API prices cached and uncached input and output separately. Search and Fetch can add request charges. | An older accepted Flash identifier can already serve a replacement. Compare behavior on the named replacement. |
| Gemini | Google’s Gemini chat app. A separate Cloud project makes Gemini API calls for your app. | Online. Gemini chat has its own activity settings. Standard unpaid API terms allow use of submitted content for product improvement. Check the region. | Gemini Apps and the developer service are hosted. | Paid use goes to the linked Cloud Billing account. That account sets the paid tier and any account cap. Model request limits apply per project. | Check the developer model you call and its supported features. The model you see in the chat picker is a separate choice. |
| Grok | Grok chat on web, mobile or X. New subscription plans can share usage across eligible Grok and Cursor features. | Online. Grok.com, Grok inside X and the developer API have separate controls and terms. | The app and developer API use hosted models. | Check the subscription’s included usage or the developer Console team’s bill for your actual route. | The developer catalog identifies the model name and availability your app can depend on. |
| Ollama | Run a supported model on your machine or choose separately billed cloud inference. | A local model runs on your machine. Cloud models process the request remotely. | Choose compatible model files and allow disk space and working memory. | Local running uses your machine. Cloud usage is metered by the model and can draw from included or purchased credits. | Keep the chosen model file or release name and settings. Check that the new version still runs and gives the expected reply. |
Does paying for chat pay for what your app uses?
Check the billing account your app actually calls. Some chat plans grant credits for a linked developer account. The developer service still decides what it charges for, which account gets the credit and what happens when it runs out.
Check whether unused credits expire and what happens at zero. For an app, price a sample request that looks like the actual task and include the reply, any attached information and paid extras. A short prompt in chat does not show what a deployed app will spend.
For example, suppose a rate card charged US$1 per million input tokens and US$4 per million output tokens. A request with 1,000 input tokens and 200 output tokens would cost US$0.0018. Five hundred requests of that size would cost US$0.90 before any other fees. Those are made-up rates. Replace both rates and token counts with figures for the model and feature your app uses.
| Charge | What to look for |
|---|---|
| Chat access | A subscription period, included usage and the price or behavior after you use it. |
| Developer API | Usually input and output tokens, the pieces a model counts as it reads and writes. Some services charge separately for search, file storage or running tools. |
| Local execution | Your machine supplies the memory, disk and running time. If you switch to a cloud model or pay a host, check that separate bill. |
Where does a prompt go, and can it be kept?
Identify the exact service and account before sending anything you need to protect. The privacy switch in a personal chat app may cover that chat only. A developer account, a work workspace or a connected service can have separate rules.
Check who gets the prompt and reply, whether they can be used for model improvement, how long they can be kept and whether someone can review them. Turning off model improvement can still leave a safety or account retention period. An export downloads the information the service offers. It does not necessarily move your projects, connected accounts or working app.
A local runner can keep model processing on the machine if you selected a local model and left cloud features off. Check what your own app saves and what the services you connect can read. For the booking draft, the model needs the allowed times and a short request. It can work without a customer’s email or a copy of the whole booking list.
Will the model fit on your computer?
Check the exact download and the memory needed while it runs. Disk holds the model files. Memory is the working space the computer uses to load a model and read the request.
Check your operating system, the memory available on your machine or graphics hardware, and the amount of text you want to send at once. Start with a short example and a model your machine can load. A downloadable model can still be too large, unsupported by your runner, or subject to a different license from the runner itself.
How can you keep one person from using the whole budget?
Give each signed-in person an allowance that your app checks before requesting a draft. The provider’s project allowance and notifications help you monitor the whole account. They may not know which visitor caused the requests. Check whether the control alerts you or blocks more usage. Even an enforced spending limit can take time to catch up.
Try this on paper or in a disposable project. For a running example, confirm that it uses a dummy signed-in account and a local stand-in that returns a fake reply. It must make no paid requests. Give that account one attempt and try two requests at the same time. Check that the app sets aside one attempt and sends exactly one request to the stand-in. If the reply goes missing, keep the attempt accounted for until you know what happened. The app can still show its own available times without asking the model again.
For a real app, keep provider credentials on the server, the part of your app that receives the request privately. Limit what one request can ask for as well as how often each person can ask. Check how the provider bills replies, optional tools and failed calls before using an allowance to estimate spend.
What can change when a model is replaced?
The name your app sends can keep working while the model answering it changes. A retired name may also stop working. Read the notice for the exact model and service your app calls.
Record the current name, the replacement and the date if the provider gives one. Keep the booking examples and expected results outside the provider. Try the replacement on those examples in a test copy before giving it the live job. Check the format, whether it invents a time and what your app will show if it cannot make a useful draft.