Skip to content

Prompt templates

Override generation framing without editing Python or changing the built-in defaults:

generation:
  system_prompt: You create a CLI risk-classification dataset.
  topic_sample_mode: bulk
  shared_scope: full
  prompt_templates:
    full_sample:
      file: prompts/bulk-full.txt

prompts/bulk-full.txt (relative to the YAML file) contains:

{{ generation_system_prompt }}
{{ instructions }}
{{ examples }}
{{ subtopics }}

Generate {{ num_items }} distinct, complete JSON records.
Return an object with an "items" list matching the supplied schema.
{{ accepted_siblings_section }}

Built-in sample and topic templates are packaged under deepfabric/templates/. Copy a template to customize its framing.

Steps and sources

Custom non-tool conversation templates supply the complete prompt; include any required message/CoT guidance yourself. Retry feedback, schema validation and exported system messages remain managed by DeepFabric. Omitted overrides keep the built-in framing.

Put template settings in the same YAML file used by deepfabric generate. The file paths below are examples, not required filenames; paths are relative to that YAML file. Inline text is also supported.

Stage / use Configuration key Example template file
Expand the topic root topics.prompt_templates.initial prompts/topic-initial.txt
Expand deeper topics topics.prompt_templates.expansion prompts/topic-expansion.txt
Independent sample or first sequential sample generation.prompt_templates.full_sample prompts/full-sample.txt
Later sequential samples generation.prompt_templates.sequential_later prompts/sequential-later.txt
Bulk seed/user-request collection generation.prompt_templates.seed prompts/seed.txt
Bulk non-tool seed completion generation.prompt_templates.completion prompts/completion.txt
Bulk full-record collection generation.prompt_templates.full_sample prompts/bulk-full.txt
Tool agent: choose each action generation.prompt_templates.agent_step prompts/agent-step.txt
Tool agent: final assistant reply generation.prompt_templates.agent_response prompts/agent-response.txt

For tool-call conversations, the sample and seed settings customize the user request, as described below. Bulk/full does not support tool-call conversations.

An omitted override retains the built-in template selected for that mode and output format. An override replaces that step's framing; include the context and history placeholders you want the model to receive. The response schema, seed pinning, validation, retries and output assembly remain controlled by code. An override for a step the configured mode never uses (for example seed in independent mode, completion for tool conversations, or agent_step without tools or with output_schema) is ignored, and loading the configuration warns which overrides are unused.

For sequential sampling, the first sample uses full_sample. Later samples use sequential_later, falling back to full_sample, then the built-in prompt. Selection follows the sample's ordinal within its topic, including resume, even when earlier samples failed.

Each override accepts inline text, {text: ...}, or {file: ...}. Files must be UTF-8. Relative paths resolve against the YAML config directory; direct Python configuration resolves them against the working directory. Sources are loaded and validated when configuration is created, so file edits during a run do not change subsequent requests. Empty text, unknown steps or placeholders, missing files, and conflicting text/file sources fail before generation.

generation.prompt_template is a shorthand for generation.prompt_templates.full_sample and cannot be combined with an explicit full_sample override. It also accepts {{{{ name }}}} placeholders.

Tool-call conversations

The agent first obtains a user request, chooses actions using structured LLM output, executes tools through Spin, and generates a final assistant reply. Templates can customize all three LLM stages independently. The action and reply overrides apply in independent, sequential, and bulk/seed sampling.

Agent stage Template setting or behavior
Independent request / first sequential request generation.prompt_templates.full_sample supplies the scenario to a fixed question-generation wrapper
Later sequential request generation.prompt_templates.sequential_later; falls back to full_sample, then the built-in prompt
Bulk user-request collection generation.prompt_templates.seed supplies the collection prompt
Start an agent for a collected request The collected request is pinned; no new question-generation call
Choose the next action / tool calls generation.prompt_templates.agent_step replaces the prompt on every action, including retries
Execute tools Spin execution; no LLM template
Write the final assistant reply generation.prompt_templates.agent_response replaces the final reply prompt

A bulk tool conversation can accept and render a completion override, but the agent uses its pinned request and the action/final-reply templates instead of that rendered prompt. generation.prompt_templates.completion therefore does not customize those tool-agent calls. It controls non-tool conversation and custom-schema completions.

The fixed question wrapper still surrounds full_sample and sequential_later scenario text in tool mode. Bulk seed overrides supply the collection prompt directly. Including raw_tools, tools, or tools_section chooses how tool information appears in that template; it does not change execution routing.

For example, extend the existing action prompt and replace the final reply prompt:

generation:
  prompt_templates:
    agent_step:
      text: |
        {{ default_prompt }}
        Prefer read operations when gathering evidence.
    agent_response:
      file: prompts/agent-response.txt

prompts/agent-response.txt could contain:

{{ dataset_system_prompt }}
User request: {{ user_request }}
Tool results: {{ tool_results }}
Give a concise answer grounded in these results.

Omitting either key preserves its existing prompt. These defaults are assembled in the agent builder; default_prompt exposes the complete default for that call, including any retry notice. Overrides change prompt text only: AgentStep and AgentResponse response schemas, tool execution, retry budgets, pinned requests, and dataset assembly stay the same. Template files are snapshotted and identified by SHA-256 and path in run reports just like other generation templates. Dry-run previews show both stages with representative action/result data.

Tool-agent fields

Both agent_step and agent_response accept the fields below. Their context is the current agent execution; sample collection fields such as num_items, seed, and sibling history belong to the earlier request-generation stage. The rendering rules below also apply here: arrays become JSON text, strings preserve whitespace, and inserted values are not interpreted as placeholders.

Placeholder Source Type Related framed field When empty
generation_system_prompt generation.system_prompt Text Required
dataset_system_prompt output.system_prompt, falling back to generation.system_prompt Text Uses fallback
raw_instructions generation.instructions Text Unset → blank; preserve whitespace
user_request Run time: generated or pinned user message Text Empty → blank
raw_tools Run time: resolved ToolDefinition dictionaries Array tools No tools → []
steps Run time: preceding AgentStep objects; all steps for the final reply Array step_history No steps → []
tool_results Run time: completed ToolExecution objects Array step_history, tool_results_text No results → []
raw_retry_feedback Run time: error passed to the current action attempt Text default_prompt No error / final reply → blank
context Run time: previous conversation passed to the final reply builder Text default_prompt Current single-turn flow / action stage → blank
Placeholder Meaning Related raw field When empty
tools Agent tool catalog with parameters and return descriptions raw_tools No tools → blank
step_history Actions, reasoning, and results by step steps, tool_results No steps → blank
tool_results_text Tool names and result summaries tool_results No results → No tools were executed.
default_prompt Complete built-in prompt for this stage and call Always set

Placeholder contract

Use {{ name }}. Substitution is literal and single-pass: no expressions, Python, loops, or conditionals. JSON braces and Docker Go-template fields such as {{.ID}} remain literal.

All placeholders render as text. Array and object values become JSON with Unicode preserved; history and seed already contain JSON text in the current implementation. Integers become their text value. Whitespace in strings is preserved. Inserted data is never parsed again as template syntax.

Run time: in a Source cell means the value is produced or selected during generation. When empty covers missing, empty, and unavailable values; it is not a universal blanking rule. Empty replacements leave surrounding template text in place. Conditional sections are not supported yet.

The raw and framed tables keep two columns visible. Use the buttons above each table to show Type, related fields, or When empty details.

Sample field availability

All six new raw fields (raw_instructions, raw_examples, raw_tools, raw_seed_field, raw_retry_feedback, and topic_path_json) are accepted in all four request/sample stages (full_sample, sequential_later, seed, completion). Tool-agent stages have their own context, listed above. Existing sample fields retain the following stage availability:

Stages Existing fields
All four generation_system_prompt, dataset_system_prompt, instructions, examples, subtopics, topic_path, tools, tools_section, num_items, seed_field, output_instructions
full_sample, sequential_later, seed additional_instructions, request_instruction, record_instructions, history, sequential_history, accepted_siblings_section, scope
completion seed, seed_section, context_guidance, retry_feedback

Accepted does not mean populated in every call. In these four stages, raw_retry_feedback receives errors only on custom-schema bulk completion retries. It also exposes errors supplied to agent_step, as described above. Unknown or unavailable stage-specific names are validation errors.

Raw and content-only sample fields

These fields expose data without added introductory prose or tags.

Placeholder Source Type Related framed field When empty
generation_system_prompt generation.system_prompt Text — Blank if empty
dataset_system_prompt output.system_prompt; fallback: generation.system_prompt Text — Empty output prompt → fallback
topic_path Run time: full target ancestry, joined with -> Text subtopics Missing or empty → blank
num_items Run time: number of output items to generate in this call Integer request_instruction Normally positive; 0 stays 0
history Run time: earlier successes for this topic JSON text sequential_history, accepted_siblings_section No history → []
scope generation.shared_scope (seed or full) Text sequential_history, request_instruction Always set
seed Run time: pinned seed or user request JSON text seed_section Preserves "", [], or null
raw_instructions generation.instructions Text instructions, additional_instructions None / "" → blank; preserve whitespace
raw_examples Run time: the same selected sample examples as examples Array examples Absent / disabled → blank; empty collection → []
raw_tools Run time: resolved ToolDefinition dictionaries Array tools, tools_section No resolved tools → blank
raw_seed_field generation.seed_field Text seed_field Unset → blank
raw_retry_feedback Run time: previous custom-schema bulk-completion validation error Text retry_feedback First attempt / other paths → blank
topic_path_json Run time: full target ancestry as an array Array subtopics Missing → blank; empty → []

Framed and formatted sample fields

These convenience fields supply wording, tags, labels, or field-name quoting. Their Meaning cells describe the purpose rather than reproducing full prompts.

Placeholder Meaning Related raw field When empty
instructions Formatted additional instructions raw_instructions None / "" → blank; whitespace keeps framing
additional_instructions Additional instructions with a label raw_instructions Empty → blank; preserve whitespace
examples Selected examples with headings and labels raw_examples Absent / disabled → blank; empty collection keeps headings
subtopics Target ancestry with introductory wording topic_path, topic_path_json Missing / completion ancestry off → blank; empty list keeps introduction
tools Readable tool names, descriptions, and parameters raw_tools No resolved tools → blank
tools_section Formatted tools inside a tagged section raw_tools No resolved tools → blank
seed_field JSON-quoted seed-field name raw_seed_field Unset → blank by default; bulk encoding differs
seed_section Pinned seed or user-request section seed Empty seed keeps framing; omitted in agent stages
sequential_history Earlier sequential successes with repetition guidance history No history / outside sequential → blank
accepted_siblings_section Accepted bulk outputs with repetition guidance history No accepted outputs / outside bulk → blank
retry_feedback Validation error with retry label raw_retry_feedback First attempt / conversation completion → blank

Generated sample instructions

These values are generated guidance, rather than raw source data.

Placeholder Purpose
request_instruction Request seeds or complete items, with count-specific wording
output_instructions Required output for this generation stage
record_instructions Record or conversation structure guidance
context_guidance How completion should use ancestry or its pinned seed

generation_system_prompt and dataset_system_prompt keep the same meaning in every stage. The system_prompt name is not accepted in sample templates. Built-in sample/seed templates use the generation prompt; completion templates use the dataset prompt, which falls back to the generation prompt.

topic_path and topic_path_json expose complete ancestry, including interior graph targets and canonical first-parent ancestry. They remain available when completion_includes_topic_path suppresses completion's subtopics framing.

raw_tools uses DeepFabric's resolved ToolDefinition dictionary shape, including parameter definitions, returns, category, and component. It is not a provider-specific tool API schema. Raw and framed examples share one selection.

Sequential history contains only earlier successful samples for the same topic: seed values in seed scope, complete records in full scope. It can be empty on a later sample if all earlier samples failed. history exposes the JSON array; sequential_history adds framing and repetition guidance. In tool mode, this history informs user-request generation; action and reply templates receive the current agent's steps and tool results instead.

sequential_history includes scope in its tag and uses scope to choose its description. request_instruction uses scope to choose wording, without inserting the literal scope value. history_section is not an accepted name.

Templates change prompt text, not API message roles. output.system_prompt and output.include_system_message continue to control exported training messages. Built-in prompt output is unchanged by choosing a custom template's raw fields.

Topic generation and expansion

topics.prompt_templates works for trees and all graph prompt_style values:

topics:
  prompt: CLI risk classification
  mode: graph
  prompt_style: anchored
  prompt_templates:
    initial:
      file: prompts/topic-initial.txt
    expansion:
      file: prompts/topic-expansion.txt

initial generates the children of the supplied root; expansion generates children of deeper nodes. The root text comes from topics.prompt, rather than a model call. Each omitted override keeps its own built-in prompt; initial is not a fallback for expansion. Sources use the same inline/file formats, validation and file snapshots as sample templates.

For example, prompts/topic-expansion.txt:

{{ system_prompt }}
Expand this topic path: {{ topic_path }}
Generate {{ num_subtopics }} focused subtopics matching the supplied schema.
Use empty connections lists for this anchored graph.

Raw and content-only topic fields

Placeholder Source Type Related framed field When empty
system_prompt topics.system_prompt Text — Empty → blank; preserve whitespace
topic_prompt topics.prompt (configured root text) Text — Root prompt is required
current_topic Run time: label of the node being expanded Text — Empty label → blank
topic_path Run time: root-to-node ancestry, joined with -> Text — Empty path → blank
num_subtopics Run time: number of children to generate Integer — 0 stays 0
num_items Run time: number of children to generate Integer — 0 stays 0
prompt_style topics.prompt_style for graphs; tree for trees Text — Always set
mode topics.mode (graph or tree) Text — Always set
raw_examples Run time: the same selected domain examples as examples Array examples Empty selection → []
topic_path_json Run time: root-to-node ancestry as an array Array — Empty path → []
raw_graph Run time: complete graph snapshot; graph mode only Object current_graph_summary Tree mode → blank
raw_tree Run time: recorded tree snapshot; tree mode only Object — Graph mode → blank; empty paths stay in snapshot

Framed and formatted topic fields

Placeholder Meaning Related raw field When empty
examples Formatted domain examples raw_examples Empty selection → blank
current_graph_summary Graph snapshot with a display-size cutoff raw_graph Tree → blank; over 20 nodes → size notice
default_prompt Complete built-in prompt for this node and style — Normal calls → nonempty

All fields in these topic tables are accepted in both initial and expansion. Sample-only fields are not accepted here. Graph ancestry follows first parents.

raw_graph uses the existing Graph.to_json() schema: nodes, root ID, and metadata. Its uncapped snapshot is constructed only when explicitly referenced; current_graph_summary retains its 20-node display cutoff.

raw_tree uses Tree.to_dict(): tree_paths, failed_generations, and config. It contains recorded completed paths, not branches still being expanded. Empty collections remain inside the snapshot. current_topic and the path fields separately describe the current expansion.

To keep the original framing, use {{ default_prompt }} and add your own instructions before or after it. A full replacement must include any desired context and connection guidance itself. Graph default prompts normally permit cross-connections; isolated and anchored prompts request empty connections lists. Templates do not change the response schema or graph processing.

Preview both stages without model calls:

uv run deepfabric generate config.yaml --topic-only --dry-run --trace-prompt --verbose

--verbose includes a deeper expansion when depth is greater than one. --topics-load skips topic generation and its templates. Run reports record the SHA-256 and file path of topic overrides under topic_prompt_templates alongside sample-template metadata.

Inspect and reproduce

uv run deepfabric generate config.yaml --topics-load topics.json --dry-run --trace-prompt --verbose --trace-prompt-file preview.prompts.jsonl

Use a fresh trace filename because tracing appends. Preview and real generation use the same template selection and rendering. Tracing records final requests, including role-bearing messages at boundaries that use message lists. Enable --run-report to save content hashes (and file paths) for overrides and built-in template assets, without their text; a config hash alone cannot capture file edits. Keep the YAML, template files, topic graph and code revision with each experiment. Resume warns if resolved templates differ from the checkpoint; use separate outputs and checkpoints for different template variants.