Tubidy

    Did Meta just figure out o1-preview's secret?!?! (Guest Lecturer)

    21:02 •
    Did Meta just figure out o1-preview's secret?!?! (Guest Lecturer)
    Download MP3

    Related Videos

    Theoretical physics of next token prediction in LLMs

    Theoretical physics of next token prediction in LLMs

    Every attention head explained

    Every attention head explained

    A new way to compare high dimensional vectors

    A new way to compare high dimensional vectors

    Rambling about GPT architecture edit ideas

    Rambling about GPT architecture edit ideas

    Simple changes can MASSIVELY improve transformers

    Simple changes can MASSIVELY improve transformers

    Can concatenated small networks compete with large ones?

    Can concatenated small networks compete with large ones?

    Curating December's best new AI papers from ArXiv

    Curating December's best new AI papers from ArXiv

    Messing with tokenization of the prompt leads to superior reasoning

    Messing with tokenization of the prompt leads to superior reasoning

    The hackiest way of making AI models self-aware

    The hackiest way of making AI models self-aware

    Can LLMs Learn by Teaching Other LLMs?

    Can LLMs Learn by Teaching Other LLMs?

    The Structured Task Hypothesis

    The Structured Task Hypothesis

    Turns out LLMs don't memorize that much

    Turns out LLMs don't memorize that much

    Autoregressive decoding of sentence vectors as opposed to tokens

    Autoregressive decoding of sentence vectors as opposed to tokens

    Some training tokens are more valuable than others

    Some training tokens are more valuable than others

    Skimming hella AI paper abstracts - Nov 5, 2024

    Skimming hella AI paper abstracts - Nov 5, 2024

    Models inside models inside models

    Models inside models inside models

    If early layers don't need tons of experts, can we save compute?

    If early layers don't need tons of experts, can we save compute?

    The END of RAG? Episodic memory for infinite context length

    The END of RAG? Episodic memory for infinite context length

    Better & Faster Large Language Models via Multi-token Prediction

    Better & Faster Large Language Models via Multi-token Prediction