Swabha Swayamdipta - What can we learn from limited observations of Language Model behavior? скачать в хорошем качестве

Swabha Swayamdipta - What can we learn from limited observations of Language Model behavior?

Talk Title: Small Samples, Big Reveal: What can we learn from limited observations of Language Model behavior? Abstract: The majority of popular language models today are both large-scale and close-sourced, making studying their behavior quite challenging. This talk tries to answer how much we can learn from limited observations of language model behavior. First, we show that language models can be reliably evaluated using even randomly selected microbenchmarks of a certain size. Second, we use language model outputs, i.e. next-token probability distributions, to build prompt inversion attacks to reveal hidden prompts with high accuracy. These findings highlight the importance of scientific research into large language models without access to large computation resources, while still allowing accountability for the providers, as well as efficient and reliable evaluation. To checkout other talks in our full NLP Seminar Series, please visit:    • UCLA NLP Seminar Series