← Back to writing

Writing

Model Customization Decision Framework

Start from a measured failure. Use prompting for instruction, structure, examples, and supplied context. Use retrieval when answers require changing external knowledge wi

Research Note

Start from a measured failure. Use prompting for instruction, structure, examples, and supplied context. Use retrieval when answers require changing external knowledge wi

Aug 4, 20261 min readBy Dalton Anderson
In this article

Model Customization Decision Framework

Start from a measured failure. Use prompting for instruction, structure, examples, and supplied context. Use retrieval when answers require changing external knowledge with provenance and access control. Consider fine-tuning when a stable task or behavior remains weak after prompt and system design and suitable labeled data exists. Consider distillation when a smaller model must reproduce defined behavior and the teacher, data, license, evaluation, and safety boundaries permit it.

Methods can be combined, but each adds data, evaluation, operations, security, privacy, and license obligations.

Choose the smallest intervention that passes a frozen evaluation set and operating constraints.

Sources

Follow the evidence.

  1. Introducing Llama 3.1ai.meta.com
  2. ai.meta.com: the llama 3 herd of modelsai.meta.com
  3. cloud.google.com: prompt design strategiescloud.google.com
  4. daltonanderson.ghost.io: unlocking llama 3 1 metas open source ai revolutiondaltonanderson.ghost.io
  5. docs.cloud.google.com: tune modelsdocs.cloud.google.com
  6. docs.cloud.google.com: rag quickstartdocs.cloud.google.com
  7. docs.nvidia.com: benchmarkingdocs.nvidia.com
  8. genai.owasp.org: owasp top 10 for llm applications 2025genai.owasp.org
  9. Meta Llama models repositorygithub.com
  10. github.com: LICENSEgithub.com
  11. github.com: MODEL CARDgithub.com
  12. github.com: USE POLICYgithub.com
  13. github.com: PurpleLlamagithub.com
  14. huggingface.co: modelshuggingface.co
  15. huggingface.co: model memory anatomyhuggingface.co
  16. open.spotify.com: 6DI2GolY5QTfflWBunE2xiopen.spotify.com
  17. opensource.org: open source ai definitionopensource.org
  18. owasp.org: www project top 10 for large language model applicationsowasp.org
  19. NIST AI Risk Management Frameworknist.gov
  20. NIST: Artificial Intelligence Risk Management Framework, Generative Artificial Intelligence Profilenist.gov
  21. youtu.be: gg2I5iO1x0oyoutu.be