Model Merging
SLURP, DeRA, pass-through, and mixture-of-experts merging are in Model Merging.
1 Bit LLM
BitNet b1.58 where every weight in a Transformer can be represented as a {-1, 0, 1} instead of a floating point number.
Resources
- Pretrained Transformers for Text Ranking: BERT and Beyond
- Principal Component Analysis
- Deep Learning AI Short Courses
- ChromaDB Tutorial on DataCamp
- Visualize Vector Embeddings in a RAG System
- Natural Language Processing Specialization on Coursera
- Distributed Representations of Sentences and Documents
- A Gentle Introduction to Doc2Vec
- Word2Vec Archive
- Mastering LLM Techniques: Inference and Optimization
- LLAMA3 Documentation
- Gensim Documentation and Examples
- TensorBoard Documentation
- Evaluation of RAG Systems
- Sentence Transformers on Hugging Face
- Local RAG with Ollama and Weaviate
- Video Lectures from ESWC 2016 on Machine Learning
- https://huyenchip.com/2023/04/11/llm-engineering.html
- https://github.com/rasbt/LLMs-from-scratch
- reasonable and good explanations of how stuff works. No hype and no vendor content
- AI by hand
- Neural Networks From Scratch
books