Tokens, Embeddings & Attention: How AI Understands Language | MIS 769 Week 05 скачать в хорошем качестве

Tokens, Embeddings & Attention: How AI Understands Language | MIS 769 Week 05

How do language models like BERT and GPT actually read text? In this lecture, we break it down from the ground up: starting with simple word counting methods (Bag-of-Words, TF-IDF), moving through Word2Vec and the leap from static to contextual embeddings, and ending with a hands-on demo showing how attention lets words "see" each other. Topics covered: Text vectorization: Bag-of-Words and TF-IDF Why early word counting methods fall short Vectors, embeddings, and the distributional hypothesis Word2Vec: CBOW and Skip-gram Static vs. contextual embeddings (and why "apple" needs more than one meaning) Vector databases and cosine similarity Tokenization methods: word, character, subword, and byte-level Live demo: attention vs. non-attention processing with BERT Part of MIS 769: Big Data Analytics for Business at UNLV's Lee Business School. Taught by Dr. Richard Young ryoung@unlv.edu