Integrative Project: Complete Solution

Proyecto integrador: desarrollo de solución completa

Desarrollo y Evaluación de Modelos Propios

Especialización en IA Generativa y Desarrollo de Negocios

Today

Agenda · 3 hours

Opening: rules, criteria, live draw of the order 10'
Presentations · round 1 (first half of the groups) 16' / group
Break 5–10'
Presentations · round 2 (second half) 16' / group
Closing: pending questions, course recap, what's next 5'+

Each slot is 15' on stage + 1' to switch screens. Time budget: 30' + 16' × groups; from 10 groups on, the break and the closing shrink to 5'.

Showcase day

Twelve sessions, one question: can you make a model your own, prove it with evidence, and explain every decision?

Opening · 10'

What we evaluate today

The question every project answers

Does our own model solve the problem better than the simplest alternative, with acceptable cost and risk?

🔧

Own model

Weights the group fine-tuned (full or LoRA) or trained, and evaluated. Prompting an API alone does not count.

⚖️

Fair comparison

Same held-out test set for the model and the baseline; decisions taken on validation, never on test.

🔍

Honest analysis

Beating the baseline is not required: a well-explained negative result beats a positive one without evidence.

Stage rules

15 minutes per group

Pitch · 8' problem → solution → evidence → value

Demo · 3' live

Q&A · 4' individual defense

🗣️ Everyone speaks

Each member presents part of the pitch or the demo. Camera on during your part and your defense.

🎯 Individual defense

At least one question to each member, about any part of the project. No AI assistants, no prepared answers.

🎲 Random order

Drawn live in a moment. Not ready when called? The group moves to the end of the list, once.

Timing: at minute 8 I ask you to switch to the demo; at minute 11 questions start. If the live demo fails for technical reasons, play your backup video (≤ 3 min).

Rules check

Minute 9: your live Gradio demo throws an error and will not come back. What now?

A technical failure of the live demo is not penalised if you switch to the backup video, and the professor later verifies that the demo works with what you delivered. Showing only the video, without trying live, caps the demo criterion at "Bueno" (below 4.5).

Individual defense

What the questions look like

“Your F1 improved after the iteration: on which split did you take that decision?”
“Why this baseline and not a stronger one?”
“Pick one error category: what is your hypothesis for it?”
“If a false negative cost twice as much, what would you change?”
“Where could personal data leak in your pipeline, and what did you do about it?”
“Tell me about one thing an AI assistant got wrong and how you caught it.”

Any member, any part. The question may be about a part you did not lead: everyone must be able to explain the whole project, including what an AI generated.

“I don't know, but I would verify it like this…” is a better answer than improvising.

Before your slot

Ready when called

💻 Tech

Screen sharing tested; pitch slides open.
Gradio: the share link only works while your notebook is running. Relaunch it shortly before your slot.
Startti: agent published, group account signed in.
Backup video link one click away.
Notifications off, camera and microphone on.

🧠 Content

2–3 demo cases: a typical one, a hard one and one where the model fails.
Who says what, and who shares the screen.
Your numbers at hand: model vs baseline, metrics, error categories, emissions.
Your recommendation in one sentence: go or no-go, and the next step.

Evaluación del proyecto · 30%

¿Cómo se califica hoy?

Socialización y demo · 10% grupal

Claridad del pitch y valor de negocio · 35%
Demo funcional · 30%
Manejo del tiempo · 15%
Participación equitativa · 20%

Defensa individual · 5% individual

Comprensión técnica de su parte y del todo · 40%
Capacidad de justificar decisiones · 35%
Uso crítico de IA · 25%

Entregable técnico · 15% grupal Se califica con lo entregado el viernes 23 de octubre a las 23:59: código, informe técnico (con el H4 en la §5), model card y registro de uso de IA.

Niveles: Excelente 4.5–5.0 · Bueno 4.0–<4.5 · Aceptable 3.0–<4.0 · Insuficiente <3.0. Solo video, sin intentar la demo en vivo: el criterio 2 queda como máximo en Bueno. Ausencia sin excusa válida: 0.0 en socialización y en defensa individual.

Evaluación del proyecto · nota individual

Tu nota del proyecto

Nota del proyecto N = (15 × T + 10 × S + 5 × D) / 30 Nota individual del proyecto NI = N × F Aporte a la nota final = 0.30 × NI F = 0.7 + 0.3 × (P − 1) / 3, con tope en 1.0

P = promedio de los puntajes (1–5) que recibes de tus compañeros de grupo en la coevaluación (sin tu autoevaluación). F se redondea a dos decimales.

P (promedio recibido)	1.0	1.5	2.0	2.5	3.0	3.5	≥ 4.0
F	0.70	0.75	0.80	0.85	0.90	0.95	1.00

T = entregable técnico: primero los descuentos por hitos (−0.3 por hito no entregado, máximo −1.2) y luego los topes (3.0 si no hay modelo propio). S = socialización y demo · D = defensa individual.

Verifica

Tus dos compañeros te asignan en promedio P = 3.5 en la coevaluación. ¿Cuál es tu factor F?

F = 0.7 + 0.3 × (3.5 − 1) / 3 = 0.7 + 0.25 = 0.95. Solo con P ≥ 4.0 el factor llega a 1.00. Y si tú no entregas tu propia coevaluación, tu F baja 0.05 (sin bajar de 0.7).

Ejemplo de la rúbrica (G03, Ana): T = 4.10, S = 4.50, D = 4.00 → N = 4.2167 · P = 3.6 → F = 0.96 → NI = 4.05.

Live draw

Presentation order

Number of groups

Skip (optional)

Seed (a number from the chat)

No draw yet.

✍ = groups whose feedback form this group's members fill in (the next two in the order). Times are minutes from the start of the session (0:00).

[0:08 · 1.5 min] Ask a student to type a 4-digit number in the chat and use it as the seed: anyone can re-run the draw with the same seed and get the same order, so the draw is verifiable. With the seed blank the deck picks a random one and shows it. Use "Skip" for group numbers that no longer exist. The order is kept in this browser (it survives a reload); "Clear" erases it. Click "Copy for chat" and paste the order (in Spanish, with start times and feedback assignments) in the chat: once groups start sharing their screens, the class no longer sees this deck. The ✍ line is the default peer-feedback assignment: every student reviews the two groups that present right after their own group (wrapping around at the end). Announce it now; you can reassign by hand if needed. The summary line shows where the break falls and whether the plan fits in 3 hours.

Coevaluación y retroalimentación · individuales

Dos formularios, hoy hasta las 23:59

Coevaluación de tu grupo cuenta para F

Confidencial: solo la ve el profesor. Calificas a cada compañero y a ti en 5 criterios (1–5); tu autoevaluación no entra en F.
Un 1 o un 2 exige justificación con ejemplos.
Obligatoria: si no la entregas, tu F baja 0.05 (sin bajar de 0.7).
PF_G<NN>_coev_<codigo> (.md o PDF) · actividad “Coevaluación del proyecto”.

Plantilla de coevaluación

Retroalimentación entre grupos formativa

No se califica. Retroalimentas a los grupos que el profesor te asignó al inicio de la sesión (✍ en el sorteo).
Una ficha por grupo, durante o justo después de su presentación.
Se consolida y se comparte con cada grupo sin nombres.
PF_retro_<codigo> (.md o PDF) · actividad “Retroalimentación entre grupos”.

Plantilla de retroalimentación

Ambos se suben a e-Aulas antes de las 23:59 del día de la Sesión 12. Buena retroalimentación: específica, basada en evidencia, útil y respetuosa.

Round 1

First half of the groups. First up: — · then —.

15' on stage + 1' switch

On stage · round 1

Now presenting

—

Next: —

Draw the order first.

[0:10 → break] Loop for every group: (1) the group shares its screen (1'); (2) Start the 15' slot timer; the phase bar follows it (pitch until 07:00 left, demo until 04:00 left, then Q&A); (3) at 07:00 left say "pasen a la demo"; (4) when the demo ends, Start the 4' Q&A timer and ask at least one question to each member, starting with whoever spoke least; (5) click "✓ Next group" (resets both timers), or "✓ Next · defense pending" if some member still owes a question: it is listed on the "Pending questions" slide. "↓ Not ready" moves the current group to the end of the list, once only. "↶ Undo" reverts the last click. Grade on the sheet during Q&A and the 1' switch. While a group shares its screen the class does not see this deck: keep it on your own screen and announce the milestones aloud and in the chat ("1 minuto de pitch", "pasen a la demo"). Go to the break slide after the last group of round 1 (shown on the draw slide).

Break

Next up: —. Round-2 groups: relaunch your demo and test your screen share now.

Round 2

Welcome back. Up now: — · then —.

15' on stage + 1' switch

On stage · round 2

Now presenting

—

Next: —

Draw the order first.

Closing the presentations

Pending questions and open floor

Groups with pending individual-defense questions. Click a group once its questions are done.

No pending questions.

One idea you will borrow

In the chat: one idea from another group's project that you would reuse in your own work, and why.

Peer-feedback forms: finish them now for the last groups, while it is fresh.

What you built in twelve sessions

From sampling one token at a time to a model of your own, evaluated, documented and demoed.

Course recap

Twelve sessions in one view

01Introduction to Generative and Pretrained Models

Generative models learn p(x) and sample from it. Prompting → RAG → fine-tuning → from scratch: more control, more cost.

02Training Generative Models

Loss, optimizer, learning rate; watch validation loss. A VAE trained on Fashion-MNIST.

03Pretrained Models and Transfer Learning

Attention and self-supervised pretraining; transfer with embeddings, few-shot or fine-tuning.

04Fine-Tuning Generative Models

Full fine-tuning vs LoRA (W = W₀ + BA): adapt a small model with curated examples.

05Evaluating Generative Models

BLEU, ROUGE, chrF, perplexity, F1 and error analysis: no single metric is enough.

06Introduction to Basic Application Development

From model to product: Gradio apps, confidence thresholds, human handoff, a first Startti agent.

07Implementing Basic Chatbots and Agents

From rules to LLM agents: conversation design, prompt injection, test conversations.

08Building Classifiers with Generative Models

Generative vs discriminative classifiers, zero-shot by label likelihood, synthetic data and its risks.

09Use Cases: Generative Models in Real Contexts

Value × feasibility, KPIs and ROI; documented failures teach as much as successes.

10Ethics and Sustainability in Generative Models

Bias, privacy (Ley 1581), regulation and carbon; model cards make them visible.

11Evaluation and Continuous Model Improvement

Error analysis → hypothesis → change → decision; robustness tests, quantization, monitoring.

12Integrative Project

Today: your own model, compared with a baseline, shown in a demo and defended.

Foundations · Applications · Responsible & continuous · click a card to flip it.

Integration

Eight minimums, eleven sessions

1 · Own model

Fine-tuned or trained by the group

S02 S03 S04 S08

2 · Baseline

Same test set, simple alternative

S03 S05 S08

3 · Two metrics

One technical, one tied to business or risk

S05 S09

4 · Error analysis

Categories, counts, hypotheses

S05 S11

5 · Iteration

Hypothesis → change → result → decision

S11

6 · Ethics & carbon

Bias, privacy, misuse, emissions

S10

7 · Model card

Mitchell et al. (2019)

S10

8 · Working demo

Gradio app or Startti agent

S06 S07

Which minimum was hardest for your group?

One word in the chat. It tells the next cohort where to start early.

What's next

Learning paths

🤗 Hugging Face courses

LLM Course (also in Spanish) and Agents Course: keep working with open models.

huggingface.co/learn

⚡ fast.ai

Practical Deep Learning for Coders: a code-first path through training and deployment.

course.fast.ai

🎓 DeepLearning.AI

Short courses on focused topics: RAG, agents, evaluation, fine-tuning.

deeplearning.ai/short-courses

🏛️ Stanford CS224N & CS230

Public materials: Transformers and NLP theory (CS224N), deep-learning practice (CS230).

CS224N · CS230

🧩 Startti ADP docs

Turn your model into an agent: knowledge bases, tools, publishing and API.

adp.startti.ai/docs

🌎 In Spanish: SomosNLP

Spanish-speaking NLP community, with hackathons and open resources.

somosnlp.org

What's next

Keep your project alive

Publish the model card, and the model if the data allow it, on the Hugging Face Hub.
Write a one-page go / no-go memo for your organization, with your numbers.
Re-run your evaluation on fresh data in three months: data drift is real.
Add the project to your portfolio, without private data or keys.

A prototype becomes a product when someone owns the metric, the data and the risk. Name that person before you pitch it internally.

Exit question

One sentence in the chat: what will you do differently at work because of this course?

Antes de las 23:59 de hoy

Lo que falta por entregar

Qué	Archivo	Actividad en e-Aulas	¿Cuenta?
Coevaluación (individual y confidencial)	`PF_G<NN>_coev_<codigo>`	Coevaluación del proyecto	Obligatoria: sin ella, tu F baja 0.05
Retroalimentación entre grupos (una ficha por grupo asignado)	`PF_retro_<codigo>`	Retroalimentación entre grupos	Formativa, no calificada

Formato: .md o PDF. Plazo: 23:59 del día de la Sesión 12. Las notas y la retroalimentación se publican en e-Aulas.

Thank you

Twelve sessions ago we asked what it takes to make a model your own. Today you showed it, with evidence.

Gracias por su trabajo, su curiosidad y sus preguntas.

References

Sources and resources

Course project kit: project brief · rubric · course assessment.
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model cards for model reporting. FAT* '19.
Gebru, T., et al. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92.
Amershi, S., et al. (2019). Guidelines for human-AI interaction. CHI 2019.
Huyen, C. (2022). Designing machine learning systems. O'Reilly.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

Hugging Face Learn: huggingface.co/learn · LLM Course in Spanish: huggingface.co/learn/llm-course/es
fast.ai, Practical Deep Learning for Coders: course.fast.ai
DeepLearning.AI short courses: deeplearning.ai/short-courses
Stanford CS224N: web.stanford.edu/class/cs224n · CS230: cs230.stanford.edu
Startti ADP documentation: adp.startti.ai/docs
SomosNLP: somosnlp.org