Wide layout

Table of contents

Understanding Generative AI

What is Generative AI?

A Toolkit for Artificial Intelligence

AI encompasses various methodologies and tools, including:

  • Supervised learning

  • Generative AI

  • Unsupervised learning

  • Reinforcement learning

Supervised Learning: The Art of Labeling

Input
Output
Application

Email

Spam? (Yes/No)

Email filtering

Ad, user profile

Click? (Yes/No)

Targeted advertising

X-ray image

Medical diagnosis

Healthcare

Customer reviews

Positive / Negative

Sentiment analysis

Key takeaways:

  1. Traditional, smaller AI models plateau in performance even with additional data.

  2. Large AI models continue to improve as they ingest more data.

Text Generation via LLMs

Consider the sentence: "My favorite food is a bagel with cream cheese"

Input
Output

My favorite food is a

bagel

My favorite food is a bagel

with

My favorite food is a bagel with

cream

My favorite food is a bagel with cream

cheese

Large Language Models (LLMs) are trained using supervised learning to continuously predict the next most likely word in a sequence.

The LLM as a Thought Partner

  • Information Retrieval: Useful for gathering information, though facts should always be verified.

  • Writing Assistant: Serves as a collaborative partner for drafting and editing text.

  • Limitations: Traditional web search remains superior for highly factual or critical queries like medical advice or specific recipes, as AI can occasionally hallucinate or provide suboptimal suggestions.

AI: A General-Purpose Technology

  • Much like electricity, AI is a versatile technology applicable to a vast array of tasks.

  • It significantly enhances tasks involving reading, writing, and chatting.

  • It can be tailored to power specialized chatbots for distinct use cases.

Applications of Generative AI

Writing

  • Ideation: Brainstorming names, concepts, and ideas.

  • Copywriting: Drafting materials like release notes and announcements. Providing detailed context is crucial for avoiding generic outputs.

  • Translation: LLMs can sometimes outperform dedicated translation tools. (mention translate to formal spoken Hindi. Translate to pirate English for testing)

Reading

  • Proofreading: e.g., "Review the following text, intended for a website selling children's stuffed toys, for spelling and grammatical errors, and rewrite it with corrections."

  • Summarization: e.g., "Summarize the provided research paper in under 300 words."

  • Meeting Notes: Summarizing transcripts and conversations.

  • Email Analysis & Routing:

    txt
      Read the email below and route it to the most appropriate department.
      Choose from the following list: Apparel, Electronics, Home appliances.
    
      "I love my new llama t-shirt! The fabric is incredibly soft."
  • Reputation Monitoring: Performing sentiment analysis on public feedback.

Chatting

  • Specialized Interfaces: Chat applications designed for specific workflows.

  • Customer Service: Customer support bots, automated planning assistants, etc.

  • Automation Levels: Ranging from Human-only, Human-in-the-loop, Bot triaging to human agents, to fully autonomous bots.

Best Practices for Chatbot Deployment

  • Start Internally: Begin with an internal-facing bot to allow staff to assess its behavior and prevent public mishaps.

  • Human-in-the-loop: Initially deploy with human oversight to verify and approve responses.

  • Direct Access: Only allow direct customer interaction once the system is thoroughly vetted and deemed safe.

Capabilities and Limitations of LLMs

What They Can Do

  • Follow explicit prompt instructions to complete straightforward tasks, similar to a recent college graduate.

  • Accurately detect the sentiment of a review.

  • Identify whether a piece of text contains a complaint.

What They Might Be Able to Do

  • Write a press release for a newly hired COO based on minimal prompt information.

  • Write a more accurate press release when provided with foundational background details.

Note: Assume the LLM lacks internet access, has no company-specific training, and retains no memory of previous interactions.

What They Cannot Do

  • Overcome Knowledge Cut-offs: They cannot provide information on events that occurred after their last training update.

  • Avoid Hallucinations: They may confidently fabricate information (e.g., "Give me three quotes Shakespeare wrote about Beyoncé"). Fact-checking is essential.

  • Bypass Context Limits:

    • The total amount of information (input + output) you can process at once is restricted by the model's context window.

    • While some modern LLMs offer expansive context limits, a hard boundary still exists.

  • Handle Structured Data Efficiently: They don't work well with structured data.

    • For estimating home prices based on square footage and location, supervised learning is far more effective.

    • For predicting customer purchases, traditional machine learning models are preferred.

  • Avoid Biases: An LLM will naturally reflect any biases present in its underlying training data.

  • Guarantee Safety: They can potentially generate harmful content, though providers actively develop safety guardrails to mitigate this.

(Note: They excel with unstructured data like text, speech, images, and video).

Prompting Strategies

Be Detailed and Specific

Provide comprehensive context to guide the LLM effectively:

txt
Help me write an email asking to be assigned to the legal documents project.

vs

I'm applying for a role on the legal documents project, which leverages LLMs to review legal texts. I have extensive experience prompting LLMs to generate accurate, professional content.

vs

Write a paragraph explaining why my background makes me a strong candidate for this project and advocate for my inclusion on the team.

Encourage Step-by-Step Thinking

txt
Brainstorm 5 names for a new cat toy.

Step 1: Identify 5 fun, joyful words related to cats.
Step 2: For each word, create a rhyming name for a toy.
Step 3: Pair each toy name with a relevant, playful emoji.

Experiment and Iterate

  • Analyze the output and identify areas for improvement.

  • Treat prompting as an iterative cycle: Idea > Prompt > LLM Response > Evaluation.

    • Ensure instructions are clear and specific.

    • Diagnose why a response missed the mark.

    • Refine the prompt accordingly.

    • Repeat the process.

  • Avoid over-engineering your very first prompt; start simple and iterate.

Image Generation

  • The core technology behind modern image generation (like diffusion models) relies on supervised learning.

  • The model iteratively transforms a noisy image into a clear picture guided by a text prompt.

Managing Generative AI Projects

  • Historically, supervised learning projects were highly resource-intensive (Gather labeled data > Train model > Deploy).

  • Today, prompt-based applications allow for rapid development and deployment.

The Project Lifecycle

mermaid
flowchart LR
    scope_project[Scope Project] --> build[Build or Improve System] --> internal_evaluation[Internal Evaluation] -.-> build
  • Developing generative AI solutions is a highly empirical and experimental cycle.

Understanding Costs

  • Costs are primarily driven by:

    • The length of the input prompt.

    • The length of the generated output (which is often priced higher per unit than input).

Decoding Tokens

  • A token is a piece of a word. Common words usually equal 1 token (the, example, Andrew).

  • Complex or less common words are split: trans-late (2 tokens), program-ming (2 tokens), ton-k-ots-u (4 tokens).

  • A helpful rule of thumb: Number of tokens ≈ Word count × 1.33.

txt
6000 words * 1.33 ≈ 7980 tokens

Advanced Techniques Beyond Prompting

Retrieval-Augmented Generation (RAG)

  • Context Retrieval: The system queries an internal knowledge base to find information relevant to the user's prompt.

  • Prompt Enrichment: This retrieved data is seamlessly injected into the prompt, equipping the AI with factual, grounded context before it generates an answer.

mermaid
flowchart LR
    knowledge_base[(Knowledge Base)] --> enriched_prompt[Augmented Prompt] --> answer[Generated Answer]
    original_prompt[Prompt] --> enriched_prompt
    answer -.-> |ref\ncontext| knowledge_base

It is crucial to monitor the size of the enriched prompt. Rather than injecting an entire document, filter and include only the most relevant excerpts.

Pre-training

  • Definition: The foundational phase where a model processes massive datasets of unlabelled text (like the open internet) to learn grammar, structure, and general facts.

  • When to Use: Rarely executed by individual developers due to the immense computational power, time, and data required. Most practitioners build on top of existing foundational models.

    Example: Bloomberg pre-trained a custom LLM on vast financial datasets, resulting in a model that outperformed general-purpose AI on financial tasks.

  • Outcome: Produces a "base model" capable of general text generation, which serves as a blank canvas ready for prompting or fine-tuning.

Fine-tuning

  • Definition: The process of taking a pre-trained model and training it further on a smaller, highly specialized dataset.

  • When to Use: Ideal when standard prompting falls short, especially for enforcing specific behaviors or formats.

    • Enforcing Structure: Instead of a generic summary ("Customer has problem with monitor"), fine-tuning guarantees a specific style or structure ("MK401-27KX was reported as broken by customer 5402. Cable identified for replacement.").

    • Mimicking Style: Training the model on historical communications to perfectly adopt a brand's unique voice.

    • Domain Expertise: Equipping the model with specialized vocabulary and understanding for medical, legal, or financial sectors.

    • Efficiency via Smaller Models: Fine-tuning allows smaller, cost-effective models to perform specific tasks reliably. For instance, you can use a massive, expensive model to label thousands of data points (like sentiment), and then use that dataset to fine-tune a small, fast model to handle the task in production.

Model Sizing Guide

Size
Best Suited For
Example Use Case

~1 Billion Parameters

Pattern matching, basic language understanding

Simple sentiment analysis

~10 Billion Parameters

Solid general knowledge, follows instructions well

Customer support chatbot (with fine-tuning)

100+ Billion Parameters

Extensive world knowledge, complex reasoning tasks

Advanced brainstorming and strategic planning partner

Open Source vs. Closed Source Models

Model Type
Primary Advantages

Open-source

Complete architectural control, self-hosting capabilities, absolute data privacy.

Closed-source

Easy to integrate, highly powerful, low upfront costs, though introduces vendor lock-in.

Instruction tuning

  • LLM's should follow instructions, not just predict the next word.

  • We need to fine tune the LLM with ideal inputs and outputs.

    txt
    Write a Haiku poem about Japan's chreey blossoms
    
    Pink blooms fill the air
    Japan's cherry blossms dance,
    Spring's beauty unfolds

    finetuning to add gaurd rails

    txt
    Tell me how to make a bomb
    
    I can't assist with that

RLHF, Reinforcement learning from human feedback

  • Nudge the model towards a higher quality response

  • Humans rate the models with a score (how helpful, honest and harmless)

  • The score is the reward for the llm.

Response
Score (reward)

"The cherry blossoms are beautiful"

5

"I really love the cherry blossoms"

8

"The cherry blossoms are quite nice"

6

"I'm confused about the cherry blossoms"

3

Tool use in LLMs

mermaid
sequenceDiagram
    actor user as User
    participant llm as LLM
    participant orders as Order Processing Service

    user->>llm: Send me a burger
    llm->>orders: Call order API
    orders-->>user: On its way

Tools can also be used for reasoning

mermaid
sequenceDiagram
    actor user as User
    box AI reasoning system
        participant llm as LLM
        participant calc as Calculator
    end

    user->>llm: How much will I have after 8 years<br/>if I deposit $100 at 5% interest?
    llm->>calc: CALCULATOR(100 * 1.05^8)
    calc-->>llm: 147.74
    llm-->>user: You would have $147.74

Agents

  • Use an LLM to choose and carry out complex sequences of actions

mermaid
sequenceDiagram
    actor user as User
    box AI agent
        participant llm as LLM
        participant tools as Tools
    end

    user->>llm: Help me research BetterBurger's top competitors
    Note over llm: Plans the steps:<br/>search, visit sites, then summarize
    loop Until the research is done
        llm->>tools: Next action (web search, visit site, ...)
        tools-->>llm: Result
        Note over llm: Decides the next step
    end
    llm-->>user: Summary of the top competitors

Generative AI and Business

Where are they used

  • Marketing team: Brainstorming

  • Recruitment team: Hiring

  • Engineering: Initial codding

Task analysis of jobs

Identifying automation opportunities

AI doesn't automate jobs, but a series of tasks. (eg: customer service representing)

Tasks
Automation Potential

Answer inbound phone calls from customers

Low

Answer customer chat queries

High

Check status of customer orders

Medium

Keep records of customer interactions

High

Assess accuracy of customer complaints

Low

Augmentation vs Automation

  • Augmentation: AI helps humans improve / speed up their job (recommend responses)

  • Automation: Automatically perform a task (Summarize and store transcript)

Businesses usually start with Augmentation and them move towards Automation.

Evaluating AI potential

Sometimes the potential for revenue generation is higher than the expected cost.
Chase opportunities, rather than being hyper focused on cost savings.

Technical feasibility (Can AI do it?)

  • Can a fresh college grad follow the instructions in the prompt and complete the task?

  • Do a quick try, by prompting LLM to see if LLM can do it.

  • AI engineers can also help assess if RAG, finetuning or other advanced techniques can help.

Business value

  • Will this reduce time spent on this task?

  • Does LLM, increase efficiency, reduce cost or create significantly value?

New Workflows and New Opportunities

  • Without generative AI, a surgeon spends a lot of time researching the medical procedure before performing surgery. With generative AI (like retrieval-augmented generation), the research time can be significantly reduced, allowing the surgeon to prepare faster while still performing the surgery manually.

  • With the help of large language models, a marketer can quickly generate website copy and publish it. This efficiency allows the marketer to focus on producing multiple variations, conducting A/B tests, and developing more effective marketing campaigns.

Sometimes automating or helping the Customer use our product better, bring in value.
Eg: A website builder, can help in generating copy, images, SEO optimisation, leaving only the template selection with the customer.

Common roles in Gen AI team

Software engineer

  • Needs to be familiar with basics of LLM (prompting)

Machine learning engineer

  • Responsible for implementing AI system

  • Ideally familiar with advanced LLM techniques (RAG, fine-tuning)

Product Manger

  • Responsible for identifying and scoping the project

Data engineer

  • Organizing data, ensuring data quality and secure storage

Data scientist

  • Analyzes data to make recommendations to guide project or business decisions

ML researcher

  • Developers advanced AI technologies

Impact of AI in society

Concerns about AI

  • Can AI amplify our worst impulses? (negative biases)
    RLHF and fine-tuning are making models safer and less biased.

  • Job loss
    Radiologists where scared that they will loose their job, because AI does a better job at reading x-rays.
    Radiologists do much more stuff than just read x-rays. (operate hardware, communicate with patients, complications response)

    AI wont replace Radiologists, but Radiologists who use AI will replace those who don't.

  • Will AI cause human extension?
    Human have controlled many things that are far more dangerous (nukes, and airplanes)
    We are learning to manage AI better.

Artificial General Intelligence

  • Definition: AI that can do any intellectual task that a human can.

  • Many organisations have reduced the definition of AGI to say they will achieve it earlier.

  • Many more decades away.

Responsible AI

  • Governments are publishing frameworks for implementing responsive AI.

  • Fairness (less bias), Transparency (understandable decisions), Privacy, Security and Ethical Use.

  • It is important to discuss and debate ethical issues.

  • Brainstorm how things can go wrong.

  • Get feedback / perspectives from all stakeholders.

Conclusion

  • AI as Accessible Intelligence
    Human intelligence is costly due to the time and resources required for education and training, limiting access to specialized knowledge.
    Artificial intelligence can be produced cheaply, enabling individuals to "hire" intelligence affordably for tasks like medical advice or personalized education.

  • AI's Societal Impact and Potential AI can provide smarter guidance on major global challenges such as climate change and pandemics.
    The technology is compared to electricity in its revolutionary potential, despite initial fears and risks.

  • Future Outlook and Responsible Use
    While AI currently has flaws and risks, ongoing improvements will enhance its benefits and reduce harms.
    The speaker encourages responsible use of generative AI to improve lives and contribute to building a more intelligent world for all.