[logseq-plugin-git:commit] 2025-12-27T19:07:25.270Z

pages/Omnivore___18-10-2023___Understanding Encoder And Decoder LLMs.md

@@ -1,7 +1,7 @@
 full-title:: [Understanding Encoder And Decoder LLMs](https://omnivore.app/me/understanding-encoder-and-decoder-ll-ms-18b4225c289)
 site:: [Ahead of AI](https://magazine.sebastianraschka.com/p/understanding-encoder-and-decoder)
 author:: Sebastian Raschka, PhD
-labels:: [[PROJECTS/SE-H2020-March-Call]] 
+labels:: [[PROJECTS/MOSAICO]] 
 date-saved:: [[18-10-2023]]
 date-published:: [[17-06-2023]]
 date-archived:: [[18-10-2023]]
@@ -9,7 +9,8 @@ is-archived:: 10
 source:: [[Omnivore]]
 state:: [[archived]]
 
-	- * ### Highlights
+	- collapsed:: true
+	  * ### Highlights
 	  collapsed:: true
 		- > BERT (**B**idirectional **E**ncoder **R**epresentations from **T**ransformers) is an encoder-only architecture based on the Transformer's encoder module. [⤴️](https://omnivore.app/me/understanding-encoder-and-decoder-ll-ms-18b4225c289#b769d9a7-30b4-414f-9fe7-b492112c397b)
 		- > The main idea behind masked language modeling is to mask (or replace) random word tokens in the input sequence and then train the model to predict the original masked tokens based on the surrounding context. [⤴️](https://omnivore.app/me/understanding-encoder-and-decoder-ll-ms-18b4225c289#7406f31a-4621-49f5-a442-0369e9b192be)