{"id":1477,"date":"2024-10-01T13:58:00","date_gmt":"2024-10-01T11:58:00","guid":{"rendered":"https:\/\/guillaume-guerard.com\/?page_id=1477"},"modified":"2024-10-09T07:37:47","modified_gmt":"2024-10-09T05:37:47","slug":"ingenierie-de-prompt-llm","status":"publish","type":"page","link":"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/","title":{"rendered":"Prompt Engineering LLM"},"content":{"rendered":"<div data-elementor-type=\"wp-page\" data-elementor-id=\"1477\" class=\"elementor elementor-1477\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a307ae8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a307ae8\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-af212c7\" data-id=\"af212c7\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-85c7493 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"85c7493\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/guillaume-guerard.com\/en\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Welcome<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-6f015a3\" data-id=\"6f015a3\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-299909d elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"299909d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Data Science<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-12f2a1a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"12f2a1a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5fc28ad\" data-id=\"5fc28ad\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3b5092b elementor-widget elementor-widget-heading\" data-id=\"3b5092b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Page contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Contents\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Les_techniques_dingenierie_de_prompts\" >Prompt engineering techniques<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Elements_de_base_de_GPT\" >Basic elements of GPT<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Les_plugins_GPT\" >GPT plugins<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Fonctionnement_des_LLMs\" >How LLMs work<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Elements_de_Prompt_Engineering\" >Elements of Prompt Engineering<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/ingenierie-de-prompt-llm\/#Optimisation_des_performances\" >Performance optimization<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Les_techniques_dingenierie_de_prompts\"><\/span>Prompt engineering techniques<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5499da4 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5499da4\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d946a98\" data-id=\"d946a98\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e7ea52e elementor-widget elementor-widget-text-editor\" data-id=\"e7ea52e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Prompt engineering has emerged as a key technique for improving the capabilities of pre-trained large-scale language models (LLMs) and visual language models (VLMs). It involves strategically designing task-specific instructions, called prompts, to guide the model\u2019s output without changing parameters. By providing a mechanism to fine-tune model outputs through carefully crafted instructions, prompt engineering enables these models to excel across diverse tasks and domains. This adaptability is different from traditional paradigms, where model retraining or extensive fine-tuning is often required for task-specific performance.<\/p><p>Sources:<\/p><p><a href=\"https:\/\/arxiv.org\/pdf\/2402.07927\" target=\"_blank\" rel=\"noopener\">https:\/\/arxiv.org\/pdf\/2402.07927<\/a><\/p><p><a href=\"https:\/\/arxiv.org\/pdf\/2310.14735\" target=\"_blank\" rel=\"noopener\">https:\/\/arxiv.org\/pdf\/2310.14735<\/a><\/p><p><img fetchpriority=\"high\" decoding=\"async\" class=\"alignnone wp-image-1483 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie.png\" alt=\"prompt ing\u00e9nierie\" width=\"685\" height=\"677\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie.png 685w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie-300x296.png 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie-12x12.png 12w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie-70x70.png 70w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/promptingenierie-125x125.png 125w\" sizes=\"(max-width: 685px) 100vw, 685px\" \/><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-4a1096b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4a1096b\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1afad73\" data-id=\"1afad73\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-63083a7 elementor-widget elementor-widget-heading\" data-id=\"63083a7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Elements_de_base_de_GPT\"><\/span>Basic elements of GPT<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a1a6c05 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a1a6c05\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a1654fe\" data-id=\"a1654fe\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7710d7b elementor-widget elementor-widget-text-editor\" data-id=\"7710d7b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>When GPT-4 receives an input prompt, the input text is first converted into tokens that the model can interpret and process. These tokens are then handled by transformation layers, which capture their relationships and context. Within these layers, attention mechanisms distribute different weights to tokens based on their relevance and context. After attention processing, the model forms its internal renderings of the input data, called intermediate representations. These representations are then decoded into human-readable text.<\/p><p>There <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/parfaire-la-methodologie\/\">method<\/a> The basic principle of prompting is to \u201cbe clear and specific.\u201d This involves formulating unambiguous and specific prompts, which can guide the model towards generating the <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/parfaire-la-section-resultats\/\">result<\/a> wish.<\/p><p>Role prompting is another fundamental method of prompt engineering. It involves giving the model a specific role to play, such as that of a helpful assistant or a knowledgeable expert. This method can be particularly effective in guiding the model&#039;s responses and ensuring that they match the desired outcome.<\/p><p>In prompt engineering, triple quotes are a technique used to separate different parts of a prompt or to encapsulate multi-line strings. This technique is especially useful when dealing with complex prompts that include multiple components or when the prompt itself contains quotes, which helps the model better understand the instructions.<\/p><p>Due to the non-deterministic nature of LLMs, it is often beneficial to try multiple times when generating answers. This technique, often called \u201cresampling,\u201d involves running the model multiple times with the same prompt and selecting the best result. This approach can help overcome the inherent variability in model answers and increase the chances of obtaining a high-quality result.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a09540a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a09540a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c43e9e4\" data-id=\"c43e9e4\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ee3b63b elementor-widget elementor-widget-heading\" data-id=\"ee3b63b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Les_plugins_GPT\"><\/span>GPT plugins<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-613591d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"613591d\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b6be236\" data-id=\"b6be236\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7f82580 elementor-widget elementor-widget-text-editor\" data-id=\"7f82580\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Before ending this discussion on prompt optimization techniques, we should mention the use of external prompt engineering assistants that have been developed recently and show promising potential. Unlike the methods presented previously, these instruments can help us refine the prompt directly. They are able to analyze user inputs and then produce relevant results in a context defined by itself, thus amplifying the effectiveness of the prompts. Some of the plugins provided by the OpenAI GPT store are good examples of such tools.<\/p><p><img decoding=\"async\" class=\"alignnone wp-image-1694 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/GPT-prompting.jpg\" alt=\"GPT prompting\" width=\"664\" height=\"436\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/GPT-prompting.jpg 664w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/GPT-prompting-300x197.jpg 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/GPT-prompting-18x12.jpg 18w\" sizes=\"(max-width: 664px) 100vw, 664px\" \/><\/p><p>Such integration can impact how LLMs interpret and respond to prompts, illustrating a link between prompt engineering and plugins. Plugins alleviate the laborious nature of complex prompt engineering, allowing the model to more efficiently understand or respond to user requests without requiring overly detailed prompts. Therefore, plugins can enhance the efficiency of prompt engineering while promoting improved user-centric effectiveness.<\/p><p>These package-like tools can be seamlessly integrated into Python and invoked directly. For example, the \u201cPrompt Enhancer\u201d plugin developed by AISEO can be invoked by starting the prompt with the word \u201cAISEO\u201d to allow the AISEO prompt generator to automatically enhance the provided LLM prompt.<\/p><p>Similarly, another plugin called &quot;Prompt Perfect&quot;, can be used by starting the prompt with &quot;perfect&quot; to automatically improve the prompt, aiming for the &quot;perfect&quot; prompt for the task at hand.<\/p><p>However, while using plugins to enhance prompts is simple and convenient, it is not always clear which prompt engineering technique, or combination of techniques, is implemented by a given plugin, given the closed nature of most plugins.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9ec5ef2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9ec5ef2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-cfaf4ae\" data-id=\"cfaf4ae\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5516827 elementor-widget elementor-widget-heading\" data-id=\"5516827\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Fonctionnement_des_LLMs\"><\/span>How LLMs work<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-16ec506 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"16ec506\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2f06929\" data-id=\"2f06929\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-60c9c51 elementor-widget elementor-widget-text-editor\" data-id=\"60c9c51\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>ChatGPT, a member of the GPT family, is built on a transformer architecture, which is at the heart of how it processes and generates answers. Transformers use a series of attention mechanisms to evaluate the importance of different parts of the input text, ensuring that the most relevant information is prioritized when generating an output.<\/p><p>The architecture consists of encoders and decoders. Encoders process the input data and capture contextual relationships between words, while decoders use this context to generate a coherent output. Self-attention is a key component, allowing the model to focus on different parts of the text based on their relevance to the task at hand. In the case of ChatGPT, once a prompt is submitted, the input text is divided into smaller units called tokens. These tokens are processed through multiple layers, which analyze their relationships to generate a final answer.<\/p><p>Additionally, ChatGPT is refined with Reinforcement Learning from Human Feedback (RLHF), where human feedback is incorporated to improve the quality of the model\u2019s responses. This process helps further align the model with user preferences and needs, ensuring that it generates results that are not only accurate, but also contextually relevant and consistent.<\/p><p>By combining transformer architecture with rapid engineering, ChatGPT excels at a variety of tasks, from casual conversation to specialized problem solving, highlighting the importance of rapid engineering to maximize the capabilities of LLMs.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6202e8a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6202e8a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a79cfb2\" data-id=\"a79cfb2\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0ffb3d9 elementor-widget elementor-widget-heading\" data-id=\"0ffb3d9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Elements_de_Prompt_Engineering\"><\/span>Elements of Prompt Engineering<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-91c761e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"91c761e\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ec479a2\" data-id=\"ec479a2\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6647ca0 elementor-widget elementor-widget-text-editor\" data-id=\"6647ca0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>At the heart of how ChatGPT and other transformer-based models work is tokenization. Tokenization involves breaking down the input text into smaller units called tokens, which can be words, subwords, or even individual characters. These tokens serve as the basic building blocks for input processing, allowing the model to analyze patterns and relationships within the data. The tokens are then processed through the transformer\u2019s attention mechanisms.<\/p><p>The self-attention mechanism is essential for understanding relationships between tokens. It allows the model to prioritize certain parts of the input based on their importance relative to the overall context. For example, in a prompt about climate change, the model would pay more attention to tokens related to the topic (e.g., \u201cglobal warming,\u201d \u201ccarbon emissions\u201d) while paying less attention to less relevant words (e.g., \u201ca,\u201d \u201cthe\u201d).<\/p><p>This self-attention mechanism allows ChatGPT to generate responses that accurately reflect the context of the prompt. By understanding how different parts of the input text relate to each other, the model can capture complex relationships, such as cause and effect, and generate logical and coherent responses.<\/p><p>GPT-4 processes prompts in multiple steps to generate relevant and accurate results. Here\u2019s a simplified description of how it works:<\/p><ol><li>Input tokenization: When a user provides a prompt, GPT-4 segments the text into smaller units that can be interpreted numerically. This process allows the model to analyze the text in a structured manner.<\/li><li>Self-attention and contextual encoding: Tokens pass through multiple layers of self-attention mechanisms. Each layer captures relationships between tokens at different levels of abstraction, helping the model understand both the local and global context of the input.<\/li><li>Intermediate representations: As tokens move through the layers of the model, GPT-4 generates intermediate representations that encode the meaning and relationships between tokens. These representations evolve as they move through successive layers, gradually building a deeper understanding of the prompt.<\/li><li>Decoding and Output Generation: Finally, GPT-4 generates output by predicting the next token in the sequence, one step at a time. The model builds on its acquired knowledge to generate consistent and contextually relevant responses, ensuring that each token is influenced by the tokens that preceded it.<\/li><\/ol><p>This process allows GPT-4 to excel at tasks such as answering <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/definir-les-questions-scientifiques\/\">questions<\/a>, perform logical reasoning, and generate detailed, human-like text. Refining inputs through attention and layered processing is critical to the model&#039;s ability to provide accurate and relevant outputs.<\/p><p>Creating effective prompts is essential to ensuring that ChatGPT provides high-quality responses. There are several key techniques for designing prompts:<\/p><p><strong>Clarity and specificity:<\/strong> The clearer and more specific a prompt is, the better the model can understand and respond appropriately. Ambiguous or overly broad prompts often result in vague or irrelevant responses. For example, instead of asking \u201cTell me about the technology,\u201d a more specific prompt like \u201cExplain how blockchain technology is used in finance\u201d will generate a more focused and informative response.<\/p><p><strong>Role Invitation:<\/strong> Assigning a specific role to ChatGPT in the prompt can significantly improve the relevance and quality of its response. For example, specifying \u201cYou are an expert historian\u201d in the prompt will cause the model to generate a more detailed and contextually rich response with historical information. Role prompting is particularly useful for specialized tasks, such as providing technical explanations or simulating professional roles (e.g., doctor, teacher, software engineer).<\/p><p>Effective prompt design also involves structuring inputs in ways that help the model generate more accurate and nuanced responses. Here are some strategies:<\/p><ol><li>Contextual inputs: Adding relevant context to the prompt can help guide the model towards a more specific answer. For example, instead of simply asking \u201cHow does climate change affect agriculture?\u201d, adding context such as \u201cIn the context of developing countries, how does climate change affect agricultural production?\u201d ensures a more targeted and detailed answer.<\/li><li>Layering information: When processing complex queries, structuring the prompt into sections can improve the model output. For example, a prompt such as \u201cExplain the main factors driving inflation, with a focus on monetary policy, supply chain disruptions, and labor market trends\u201d ensures that the model addresses each component in a structured and logical way.<\/li><li>Iterative prompts: For tasks that require refinement, using iterative prompts can lead to better results. Users can refine the prompt based on previous model output, guiding it step-by-step through complex explanations or problem-solving tasks. This approach is particularly useful for multi-step reasoning or technical tasks.<\/li><\/ol><p>By using these techniques, users can significantly improve the quality of ChatGPT results, making the responses more tailored to their specific goals.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f50ee5b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f50ee5b\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c616182\" data-id=\"c616182\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b639648 elementor-widget elementor-widget-heading\" data-id=\"b639648\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Optimisation_des_performances\"><\/span>Performance optimization<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-880ec00 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"880ec00\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ef9146b\" data-id=\"ef9146b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-cd1cb1c elementor-widget elementor-widget-text-editor\" data-id=\"cd1cb1c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Evaluating the results of advanced incentive techniques is essential to ensure that the results generated by the model are accurate, reliable, and useful for the task at hand. Several evaluation methods can be applied depending on the nature of the task. <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/\">research<\/a>\u00a0:<\/p><p><strong>Accuracy and relevance\u00a0<\/strong>: For factual tasks, such as synthesizing research or answering specific questions, the accuracy of the results should be checked against reliable sources. This is especially important for zero- or few-trial tasks, where the model relies heavily on pre-existing knowledge.<\/p><p><strong>Logical consistency\u00a0<\/strong>: When using the chain of thought or self-consistency prompt, it is important to assess the logical flow of the reasoning. Does the result make sense step by step? For research tasks that involve multiple-step reasoning, make sure that each step is valid and contributes to the <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/parfaire-la-conclusion\/\">conclusion<\/a> global.<\/p><p><strong>Consistency<\/strong>\u00a0: In the self-consistency prompt, assess the consistency of multiple model results. If multiple responses converge to the same answer, this increases confidence in the accuracy of the result. If there is significant variation, it may be necessary to refine the prompt or task.<\/p><p><strong>Completeness<\/strong> : Especially in the case of a CoT prompt, make sure that the model addresses all parts of the task. For example, if the prompt asks the model to explain both natural and human drivers of climate change, the answer should cover both aspects comprehensively.<\/p><p><strong>Bias and objectivity<\/strong> : assess whether the model has introduced bias into its responses, especially for sensitive research topics. The model should remain objective and refrain from giving biased opinions unless the task specifically requires subjective input.<\/p><p><strong>User testing<\/strong> : For tasks requiring complex reasoning, such as solving mathematical or scientific problems, results can be evaluated by subject matter experts or through user studies to assess their validity and usability.<\/p><p>By applying these evaluation methods, users can ensure that advanced prompting techniques produce reliable and high-quality results for various research tasks.<\/p><p>Optimizing prompt performance in large language models (LLMs) like GPT-4 is critical to ensuring that answers are accurate, relevant, and consistent with user expectations. By adjusting how the model interprets inputs and generates outputs, users can exercise greater control over the quality of answers. Two key strategies for optimizing prompt performance include: <strong>temperature<\/strong> And <strong>sampling<\/strong> and the use of contextual information.<\/p><p><strong>Temperature :<\/strong> This parameter controls the randomness of the model output. The scale typically ranges from 0 to 1.<\/p><p>Lower temperatures (closer to 0) produce more targeted and deterministic outputs. The model is more likely to choose the most likely next word, leading to accurate and consistent answers. This is ideal for factual, technical, or formal writing where accuracy and clarity are essential.<\/p><p>Higher temperatures (closer to 1) introduce more diversity and randomness into the output. The model is less constrained by probability, which encourages creativity and variability in responses. This setting works well for creative tasks, such as story generation, brainstorming, or open-ended questions.<\/p><p>Example :<br \/>Prompt: \u201cWrite a formal explanation of quantum mechanics.\u201d<br \/>With temperature set to 0.2, the model will provide a clear and accurate explanation based on high probability tokens.<br \/>With a temperature of 0.8, the response may include more varied, potentially creative or unconventional explanations that could be useful for exploring different perspectives.<\/p><p><strong>Top-p (kernel) sampling:<\/strong> In Top-p sampling, the model selects the highest percentage of likely next words rather than strictly following the highest probability path. The model continues sampling until the cumulative probability reaches a user-defined threshold (p).<\/p><p>Lower values of p (e.g., 0.5) lead to more conservative responses, while higher values (e.g., 0.9) introduce more variety into the output.<\/p><p>Example :<br \/>Prompt: \u201cDescribe potential future applications of artificial intelligence in healthcare.\u201d<\/p><p>At a higher p-value of 0.5, the model will provide standard and highly probable responses, such as improvements in diagnostic tools and personalized treatments.<\/p><p>With a higher p-value of 0.9, the model will include more diverse and possibly speculative responses, such as AI advances in emotional support for patients or new applications in telemedicine.<\/p><p>In many cases, the performance of a prompt can be significantly improved by<strong> integrating a context<\/strong> additional input. This technique guides the model more effectively by providing it with relevant background information or framing the prompt in a specific way that leads to more targeted and accurate results.<\/p><p>Contextual prompts are designed to include external data, background knowledge, or specific instructions that help guide the model toward a more specific answer. For example, instead of asking a general question like \u201cHow important is renewable energy?\u201d, incorporating context into the prompt, such as a geographic focus or a specific time period, can lead to more targeted and relevant answers.<\/p><p>Example :<br \/>Prompt: \u201cAccording to the IPCC 2022 report, explain how renewable energy contributes to reducing carbon emissions in developing countries.\u201d<\/p><p>The inclusion of the IPCC report in the prompt restricts the scope of the model, causing it to rely on specific knowledge about carbon emissions and renewable energy trends in developing countries.<\/p><p>By incorporating such contextual clues, users can tailor responses to their specific needs, making the model more responsive to complex or specialized requests.<\/p><p>Another effective strategy is to <strong>assign roles<\/strong> to the model, in combination with built-in context. For example, asking ChatGPT to \u201cact as a climate scientist\u201d when answering a question about renewable energy ensures that the result is framed with the appropriate expertise and guidance.<\/p><p>Example :<br \/>Prompt: \u201cAs a climate scientist, explain the long-term environmental benefits of wind energy in Europe, focusing on the last decade.\u201d<\/p><p>The role-based prompt gives the model clear guidance on the expertise expected in the response, while the built-in context focuses it on a specific geographic area and time period.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Home Data Science Prompt Engineering Techniques Prompt engineering has become an essential technique for improving the capabilities \u2026 <\/p>","protected":false},"author":1,"featured_media":0,"parent":47,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"elementor_header_footer","meta":{"footnotes":""},"class_list":["post-1477","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1477","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/comments?post=1477"}],"version-history":[{"count":22,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1477\/revisions"}],"predecessor-version":[{"id":1698,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1477\/revisions\/1698"}],"up":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/47"}],"wp:attachment":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/media?parent=1477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}