AI Text Generation — Trained on Tolstoy's "La Guerre et la Paix" · Transformer built from scratch

Connecting...
—
Parameters
—
Vocabulary
—
Layers
—
Heads
—
Embed Dim
✦ Prompt
0.8
300
🏗 Αρχιτεκτονική Μοντέλου
Input
Character-level tokenizer → token IDs
↓
Embeddings
Token Embedding + Position Embedding (128-dim)
↓
Transformer Block × 4
Pre-LayerNorm → Multi-Head Attention (4 heads) → Residual
Pre-LayerNorm → FeedForward (GELU, 4× expansion) → Residual
↓
LayerNorm + LM Head
Final normalization → Linear projection → vocab logits
↓
Output
Softmax sampling with temperature → generated text