Key parameters
Temperature
Controls the degree of randomness in token selection.
- Higher temperatures result in a higher number of candidate tokens from which the next output token is selected, and can produce more diverse results.
- A temperate of 0 results in greedy decoding, selecting the most probably token at each step
Top-P
Controls the diversity of model's ouputs.
- Top-P defines the probability threshold that, once cumulatively exceeded, tokens stop being selected as candidates.
- A top-P of 0 is typically equivalent to greedy decoding.
- A top-P of 1 typically selects every token in the model’s vocabulary.
Top-k
INFO
- Top-K is a positive integer that defines the number of most probable tokens from which to select the output token. A top-K of 1 selects a single token, performing greedy decoding.
For further reading, check out
- # Mastering AI Creativity: A Guide to Temperature, Top-K, Top-P
- # Understanding OpenAI Parameters: Optimize your Prompts for Better Outputs
Prompting Techniques
Prompt Engineering helps to effectively design and improve prompts to get better results on different tasks with LLMs.
Meta Prompting
Self-Consistency
Generate Knowledge Prompting
Prompt Chaining
Tree of Thoughts
Retrieval Augmented Generation
Automatic Reasoning and Tool-use
Automatic Prompt Engineer
Active-Prompt
Directional Stimulus Prompting
Program-Aided Language Models
ReAct
Reflexion
Multimodal CoT
Graph Prompting