NLP-CV.AJ1
Transformers for Natural Language Processing and Computer Vision
Master Transformers for NLP and CV, from architecture to Generative AI with GPTs, ViT, and Stable Diffusion. Build, fine-tune, and deploy.
- Practice in 27 Hands-On Labs — nothing to install
- 21 Interactive Lessons and 135 topics mapped to the official exam objectives
Intermediate Self-paced · 1 year access
27 Hands-On LiveLabs
Practice real IT tasks in guided environments.
- Real environments
- Auto-graded
- No installation
01 / Skills you'll get
What you will be able to do
Transformer Architecture Mastery: Deeply understand the 'Attention Is All You Need' paradigm, encoder-decoder structures, and how to implement foundational Transformer Models like BERT and RoBERTa from scratch, including their pretraining and fine-tuning nuances.
Generative AI Development: Gain practical expertise in leveraging and fine-tuning cutting-edge Generative AI models such as OpenAI GPTs (GPT-4, RAG), T5 for summarization, and exploring advanced LLMs like PaLM 2, understanding their capabilities and inherent limitations.
Computer Vision with Transformers: Develop proficiency in applying Vision Transformer (ViT) models, CLIP, and DALL-E for multimodal tasks, and master text-to-image generation with Stable Diffusion, including automated prompt design and training vision models without coding via Hugging Face AutoTrain.
Advanced Deployment & Risk Mitigation: Learn to interpret transformer behavior using tools like BertViz and SHAP, implement LLM embeddings as an alternative to fine-tuning, and critically assess and mitigate risks associated with large language models, paving the way for functional AGI.
Course Highlights
-
21 Structured Lessons Comprehensive coverage of core course objectives
-
27 Hands-On LiveLabs Interactive guided scenarios with instant evaluation
-
1 Year Full Access Self-paced learning accessible anytime on all devices
02 / Lessons & labs
See exactly what you will learn and practice
Lessons
21 Interactive Lessons · 135 topics01 Preface 2 topics +
- Who this course is for
- What this course covers
02 What are Transformers? 6 topics · 1 LiveLab +
- Foundation Models
- A brief history of how transformers were born
- The new role of AI professionals
- The rise of seamless transformer APIs
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
03 Getting Started with the Architecture of the Transformer Model 5 topics · 2 LiveLab +
- The rise of the Transformer: Attention Is All You Need
- Training and performance
- Hugging Face transformer models
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
04 Emergent vs Downstream Tasks: The Unseen Depths of Transformers 5 topics · 2 LiveLab +
- The paradigm shift: What is an NLP task?
- Investigating the potential of downstream tasks
- Running downstream tasks
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
05 Advancements in Translations with Google Trax, Google Translate, and Gemini 7 topics · 1 LiveLab +
- Defining machine translation
- Evaluating machine translations
- Translations with Google Trax
- Translation with Google Translate
- Translation with Gemini
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
06 Diving into Fine-Tuning through BERT 5 topics · 1 LiveLab +
- The architecture of BERT
- Fine-tuning BERT
- Building a Python interface to interact with the model
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
07 Pretraining a Transformer from Scratch through RoBERTa 6 topics · 2 LiveLab +
- Training a tokenizer and pretraining a transformer
- Building KantaiBERT from scratch
- Pretraining a Generative AI customer support model on X data
- Next steps
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
08 The Generative AI Revolution with ChatGPT 7 topics · 3 LiveLab +
- GPTs as GPTs
- The architecture of OpenAI GPT transformer models
- OpenAI models as assistants
- Getting started with the GPT-4 API
- Retrieval Augmented Generation (RAG) with GPT-4
- Summary
- References
3 LiveLab in this lesson — see the labs panel →
09 Fine-Tuning OpenAI GPT Models 9 topics +
- Risk management
- Fine-tuning a GPT model for completion (generative)
- Preparing the dataset
- Fine-tuning an original model
- Running the fine-tuned GPT model
- Managing fine-tuned jobs and models
- Before leaving
- Summary
- References
10 Shattering the Black Box with Interpretable Tools 6 topics · 2 LiveLab +
- Transformer visualization with BertViz
- Interpreting Hugging Face transformers with SHAP
- Transformer visualization via dictionary learning
- Other interpretable AI tools
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
11 Investigating the Role of Tokenizers in Shaping Transformer Models 4 topics · 1 LiveLab +
- Matching datasets and tokenizers
- Exploring sentence and WordPiece tokenizers to u...fficiency of subword tokenizers for transformers
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
12 Leveraging LLM Embeddings as an Alternative to Fine-Tuning 6 topics · 2 LiveLab +
- LLM embeddings as an alternative to fine-tuning
- Fundamentals of text embedding with NLTK and Gensim
- Implementing question-answering systems with embedding-based search techniques
- Transfer learning with Ada embeddings
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
13 Toward Syntax-Free Semantic Role Labeling with ChatGPT and GPT-4 9 topics · 1 LiveLab +
- Getting started with cutting-edge SRL
- Entering the syntax-free world of AI
- Defining SRL
- SRL experiments with ChatGPT with GPT-4
- Questioning the scope of SRL
- Redefining SRL
- From task-specific SRL to emergence with ChatGPT
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
14 Summarization with T5 and ChatGPT 8 topics · 1 LiveLab +
- Designing a universal text-to-text model
- The rise of text-to-text transformer models
- A prefix instead of task-specific formats
- The T5 model
- Text summarization with T5
- From text-to-text to new word predictions with OpenAI ChatGPT
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
15 Exploring Cutting-Edge LLMs with Vertex AI and PaLM 2 6 topics +
- Architecture
- Assistants
- Vertex AI PaLM 2 API
- Fine-tuning
- Summary
- References
16 Guarding the Giants: Mitigating Risks in Large Language Models 9 topics · 3 LiveLab +
- The emergence of functional AGI
- Cutting-edge platform installation limitations
- Auto-BIG-bench
- WandB
- When will AI agents replicate?
- Risk management
- Risk mitigation tools with RLHF and RAG
- Summary
- References
3 LiveLab in this lesson — see the labs panel →
17 Beyond Text: Vision Transformers in the Dawn of Revolutionary AI 7 topics · 2 LiveLab +
- From task-agnostic models to multimodal vision transformers
- ViT – Vision Transformer
- CLIP
- DALL-E 2 and DALL-E 3
- GPT-4V, DALL-E 3, and divergent semantic association
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
18 Transcending the Image-Text Boundary with Stable Diffusion 6 topics · 1 LiveLab +
- Transcending image generation boundaries
- Part I: Defining text-to-image with Stable Diffusion
- Part II: Running text-to-image with Stable Diffusion
- Part III: Video
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
19 Hugging Face AutoTrain: Training Vision Models without Coding 8 topics · 1 LiveLab +
- Goal and scope of this lesson
- Getting started
- Uploading the dataset
- Training models with AutoTrain
- Deploying a model
- Running our models for inference
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
20 On the Road to Functional AGI with HuggingGPT and its Peers 8 topics · 1 LiveLab +
- Defining F-AGI
- Installing and importing
- Validation set
- HuggingGPT
- CustomGPT
- Model Chaining with Runway Gen-2
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
21 Beyond Human-Designed Prompts with Generative Ideation 6 topics +
- Part I: Defining generative ideation
- Part II: Automating prompt design for generative image design
- Part III: Automated generative ideation with Stable Diffusion
- The future is yours!
- Summary
- References
Hands-On Labs Our edge
27 LiveLabs- Training, Evaluating, and Visualizing a Machine Learning Classifier
- Implementing Multi-Head Attention and Post-Layer Normalization
- Exploring Positional Encoding in Transformer Models
- Visualizing Decision Boundaries with k-NN Using 1000 Random Samples
- Running Downstream Transformer Tasks
- Preprocessing the WMT14 French-English Dataset and Evaluating with BLEU
- Fine-Tuning BERT for Sentence Classification Using the CoLA Dataset
- Building and Training KantaiBERT for Token Classification
- Building a Customer-Support Assistant Using a Transformer Model
- Analyzing GPT Transformer Architecture and OpenAI Model APIs
- Getting Started with OpenAI GPT-4 for NLP Tasks
- Implementing RAG Using GPT-4
- Visualizing Transformer Attention with BertViz
- Interpreting Transformer Predictions Using SHAP
- Exploring Tokenizers in Modern NLP Using HuggingFace
- Building Word Embeddings Using NLTK and Gensim
- Building an Embedding-Based Question-Answering and Transfer-Learning Pipeline
- Performing Zero-Shot SRL Using GPT-4 Via Prompting
- Building and Evaluating Text Summarization Systems
- Evaluating Auto-BIG-bench Tasks
- Evaluating and Mitigating Hallucination in RAG Systems
- Mitigating Risks in Generative AI Systems
- Exploring Vision-Language Models with CLIP and ViT
- Generating and Interpreting AI-Driven Visual Content Using GPT-4V and DALL·E
- Generating Images with Stable Diffusion Using Keras
- Training NLP Models Automatically with Hugging Face AutoTrain
- Analyzing Images Using ViT Models
03 / FAQs
Questions before you start
Who is this course designed for? +
What are the practical applications covered? +
<
p dir="ltr">You'll build and fine-tune models for machine translation, text summarization, question-answering systems, semantic role labeling, and cutting-edge text-to-image generation. We also cover integrating with APIs like GPT-4 and Vertex AI PaLM 2 for real-world Generative AI solutions.
Does this course cover the latest Transformer models? +
Absolutely. We dive into the architecture and application of current models like BERT, RoBERTa, T5, OpenAI GPTs (including GPT-4 and RAG), Vision Transformer (ViT), CLIP, DALL-E 3, Stable Diffusion, and PaLM 2, ensuring you're up-to-date with the Generative AI landscape.
What are the limitations or challenges addressed in the course? +
We explicitly address critical aspects like the trade-offs in fine-tuning vs. embeddings, the role of tokenizers in model performance, interpreting black-box models, and significant risks associated with large language models, including ethical considerations and platform limitations. Expect to learn how to debug and mitigate common failure points.
Keep going
You might also like
Ready to Build the Future of Multimodal AI?
The line between text and vision is disappearing. Start your journey to becoming a lead AI architect and master Transformers for NLP and CV to stay ahead in the rapidly evolving Generative AI landscape.
- 1 year of full access
- 27 LiveLab included
- Certificate of completion
No credit card required