Why prompting techniques are important in the era of large models
GPT-4o and Claude 3.5 Sonnet are smart, but without the right prompt they give mediocre answers on complex tasks. The difference between "answer the question" and a CoT prompt on GSM8K is 17% accuracy. That's a lot.
Chain-of-Thought (CoT)
Idea: ask the model to show its reasoning before answering. It works because intermediate steps reduce the probability of error on the next step.
# Без CoT — модель часто ошибается
prompt = "Джон купил 3 пакета по 5 яблок и съел 7. Сколько осталось?"
# С CoT — точность выше на 30%+
prompt = """Джон купил 3 пакета по 5 яблок и съел 7. Сколько осталось?
Давай решим шаг за шагом:"""
# Zero-shot CoT (работает почти так же)
prompt = """Джон купил 3 пакета по 5 яблок и съел 7. Сколько осталось?
Думай пошагово."""
ReAct: thought + action
ReAct (Reasoning + Acting) alternates reasoning with tool calls. It's a standard for agents in LangChain, AutoGen, CrewAI. Each step: Thought → Action → Observation → next Thought.
system_prompt = """Ты помощник с доступом к инструментам.
Для каждого шага:
1. Thought: что нужно сделать
2. Action: какой инструмент вызвать и с какими аргументами
3. Observation: результат вызова
4. ... повторяй до Final Answer"""
SELF-ASK: question decomposition
Technique for complex multi-hop questions. The model itself formulates subquestions and answers them before the final answer. Improves accuracy by 20-35% on tasks like 'who was the president of the country where X was invented'.
prompt = """Вопрос: В каком году основали компанию, которая создала модель с лучшим MMLU в 2023?
Нужны ли промежуточные вопросы? Да.
Промежуточный вопрос: Какая модель показала лучший MMLU в 2023?
Промежуточный ответ: GPT-4 от OpenAI.
Промежуточный вопрос: В каком году основана OpenAI?
Промежуточный ответ: 2015.
Финальный ответ: 2015."""
Comparison of techniques
| Technique | Where it works | Accuracy gain | Token cost |
|---|---|---|---|
| Zero-shot CoT | Math, logic | +25-45% | +30% |
| Few-shot CoT | Math, reasoning | +40-55% | +50% |
| ReAct | Agents, search | +30-40% | +60% |
| SELF-ASK | Multi-hop QA | +20-35% | +40% |
FAQ
Does CoT work with Russian-language prompts?
Yes. GPT-4o, Claude 3.5, Qwen 2.5 excellently understand "think step by step" and "let's solve step by step" in Russian.
Do I need to pay for additional reasoning tokens?
Yes, CoT increases answer length. For simple tasks it's not justified. For complex ones, savings from reduced retries due to errors outweigh the cost.