Lectures

In this lecture, you will build an engineering view of how to assess the quality of answers in a ChatGPT application using a second “judge” model: which criteria to capture (correctness, helpfulness, style, safety), how to write a rubric‑prompt as a quality specification, how to require strictly valid JSON from the judge for CI, how to connect eval cases with golden prompts, and which limitations (bias, positional bias, verbosity bias) you must account for in production.

In this lecture you will learn how to turn “ideal dialogues” (golden prompts) into engineering regression tests (golden cases) for LLM applications: how a case is structured (input, expectations, rubric, thresholds), where to store the sets in the repository, how to link them to versioned rubric prompts, how to run them locally and in CI, as well as how to distinguish threshold vs baseline approaches and separate safety/negative cases into a dedicated strict layer.

In this lecture, we break down how to “painlessly” embed a ChatGPT App into an existing product: we treat MCP as a thin adapter/gateway over REST/GraphQL and domain services, build a single source of truth for schemas (e.g., via Zod), design contract tests, plan versioning for tools/ resources, and run zero‑downtime migrations. We also discuss the evolution of the OAuth layer (including /.well-known/oauth-protected-resource, scopes, and WWW-Authenticate), the hint layer of metadata/annotations ( readOnlyHint, destructiveHint, openWorldHint), as well as SDK upgrade processes and common mistakes that lead to runtime incidents and quality degradation.

We break down how to turn an LLM‑App from a “frozen prompt” into a living system: assemble a signal map (tech, economics, product, quality), build a canonical feedback loop in 4 steps (signal → hypothesis → controlled change → offline/online verification), design an internal “improvements assistant” as a separate agent, discuss what can be tuned (prompt/tools/UX/economics) and where human‑in‑the‑loop is mandatory (permissions, payments, safety, data policy).

In this lecture, we break down how Multi‑App scenarios work in ChatGPT Apps: how the model “routes” requests among multiple apps, why metadata and description become your LLM SEO, how to explain the UI via _meta [ "openai/widgetDescription" ], and how annotations like readOnlyHint, destructiveHint, openWorldHint, and openai/isConsequential help you build pipelines safely. We’ll also look at composition patterns (pipeline/handoff/shared context), interoperability via JSON outputs, and common pitfalls.