Build LLM apps with APIs: tools, structured output, evals

Using an AI assistant in a browser is one thing. Building a product feature on top of a language model is another. You need predictable output your code can parse, ways for the model to look up data or take actions, protection against misuse, control over cost and latency, and some way of knowing whether a change made things better or worse.

This course teaches those skills by building Desk, an internal helpdesk assistant that answers employee questions, looks up order and account data through tools, drafts replies and escalates to a human when it should. We write Python against model provider APIs directly, so you understand what frameworks do before deciding whether to use one.

What you will learn

• How chat completion style APIs work: messages, system prompts, tokens, parameters and streaming
• Getting structured output that matches a JSON schema
• Tool calling, and the loop that turns a model into a small agent
• Designing tools that are safe, narrow and easy for a model to use
• Evaluations: test sets, graders and regression checks
• Cost, latency, caching and choosing between models
• Security: prompt injection, data exposure and permission boundaries

Vendor neutral

Examples use a thin wrapper so the same code runs with different providers' SDKs. We point out where providers differ, but the concepts are the same across them.

You should be comfortable with Python, HTTP APIs and JSON. No machine learning background is required.