{"id":1553,"date":"2024-10-02T13:38:48","date_gmt":"2024-10-02T11:38:48","guid":{"rendered":"https:\/\/guillaume-guerard.com\/?page_id=1553"},"modified":"2024-10-02T14:16:56","modified_gmt":"2024-10-02T12:16:56","slug":"self-consistency","status":"publish","type":"page","link":"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/","title":{"rendered":"Self-Consistency"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"1553\" class=\"elementor elementor-1553\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-abca5c3 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"abca5c3\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-2feeba9\" data-id=\"2feeba9\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ef59546 elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"ef59546\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/guillaume-guerard.com\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Accueil<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-00728cb\" data-id=\"00728cb\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-95e3c9e elementor-align-justify elementor-widget elementor-widget-button\" data-id=\"95e3c9e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/complex-systems-ai.com\/\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Data Science<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d854242 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d854242\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1dc6f33\" data-id=\"1dc6f33\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e18bb67 elementor-widget elementor-widget-heading\" data-id=\"e18bb67\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Contenus de la page<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/#Self-Consistency_pour_consolider_la_chaine_de_pensee\" >Self-Consistency, pour consolider la chaine de pens\u00e9e<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/#Plusieurs_reponses_pour_faire_un_cas_general\" >Plusieurs r\u00e9ponses pour faire un cas g\u00e9n\u00e9ral<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/#Code_pour_le_Self-Consistency\" >Code pour le Self-Consistency<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/#Self-consistency_sur_multiLLM\" >Self-consistency sur multiLLM<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/self-consistency\/#Self-consistency_meilleure_pratique\" >Self-consistency meilleure pratique<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Self-Consistency_pour_consolider_la_chaine_de_pensee\"><\/span>Self-Consistency, pour consolider la chaine de pens\u00e9e<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c9e29ba elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c9e29ba\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-e80debb\" data-id=\"e80debb\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2fced63 elementor-widget elementor-widget-text-editor\" data-id=\"2fced63\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>L\u2019auto-coh\u00e9rence (self-consistency) est peut-\u00eatre l\u2019une des techniques les plus avanc\u00e9es pour une ing\u00e9nierie rapide. Propos\u00e9 par Wang et al. (2022), l\u2019autocoh\u00e9rence vise \u00ab \u00e0 remplacer le d\u00e9codage na\u00eff et gourmand utilis\u00e9 dans l\u2019incitation \u00e0 la cha\u00eene de pens\u00e9e \u00bb. L&rsquo;id\u00e9e est d&rsquo;\u00e9chantillonner plusieurs chemins de raisonnement divers \u00e0 travers un CoT en quelques coups et d&rsquo;utiliser les g\u00e9n\u00e9rations pour s\u00e9lectionner la r\u00e9ponse la plus coh\u00e9rente. Cela contribue \u00e0 am\u00e9liorer les performances des invites CoT sur les t\u00e2ches impliquant un raisonnement arithm\u00e9tique et de bon sens.<\/p><p>Pour les t\u00e2ches de raisonnement complexes comportant plusieurs chemins valides, l\u2019auto-coh\u00e9rence g\u00e9n\u00e8re diverses cha\u00eenes de raisonnement par \u00e9chantillonnage \u00e0 partir du d\u00e9codeur du mod\u00e8le de langage. Il identifie ensuite la r\u00e9ponse finale la plus coh\u00e9rente en marginalisant ces cha\u00eenes \u00e9chantillonn\u00e9es. Cette approche capitalise sur l\u2019observation selon laquelle les probl\u00e8mes n\u00e9cessitant une analyse r\u00e9fl\u00e9chie entra\u00eenent souvent une plus grande diversit\u00e9 de raisonnement, conduisant \u00e0 une solution.<\/p><p>La combinaison de l&rsquo;auto-coh\u00e9rence et de la cha\u00eene de pens\u00e9e entra\u00eene des am\u00e9liorations significatives de la pr\u00e9cision sur divers benchmarks, tels que 17,9 % sur GSM8K, 11,0 % sur SVAMP, 12,2 % sur AQuA, 6,4 % sur StrategyQA et 3 \u00e0 9 % sur D\u00e9fi ARC par rapport \u00e0 l\u2019incitation \u00e0 la cha\u00eene de pens\u00e9e de base.<\/p><p><img decoding=\"async\" class=\"alignnone wp-image-49 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2023\/07\/logo_sc.png\" alt=\"self-consistency\" width=\"238\" height=\"203\" \/><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-53a27ba elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"53a27ba\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-be44d66\" data-id=\"be44d66\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a7c4a4d elementor-widget elementor-widget-heading\" data-id=\"a7c4a4d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Plusieurs_reponses_pour_faire_un_cas_general\"><\/span>Plusieurs r\u00e9ponses pour faire un cas g\u00e9n\u00e9ral<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-dc3fe99 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"dc3fe99\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ffa5832\" data-id=\"ffa5832\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f24451e elementor-widget elementor-widget-text-editor\" data-id=\"f24451e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>L&rsquo;auto-coh\u00e9rence est une approche qui demande simplement \u00e0 un mod\u00e8le la m\u00eame invite plusieurs fois et prend le <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/parfaire-la-section-resultats\/\">r\u00e9sultat<\/a> majoritaire comme r\u00e9ponse finale. Il s&rsquo;agit d&rsquo;un suivi des invites CoT et est plus puissant lorsqu&rsquo;il est utilis\u00e9 conjointement avec celui-ci.<\/p><p>Prenons un exemple simple d&rsquo;analyse d&rsquo;e-mails. Supposons que vous soyez un \u00e9diteur de logiciels et que vous recevez des centaines d\u2019e-mails par jour. Vous souhaitez utiliser un mod\u00e8le pour classer les e-mails comme importants ou non, afin de pouvoir prioriser ceux qui peuvent avoir un impact majeur sur votre entreprise.<\/p><p>Voici un exemple d&rsquo;e-mail que vous pourriez recevoir. Mettons cela dans une invite\u00a0:<\/p><p><img fetchpriority=\"high\" decoding=\"async\" class=\"alignnone wp-image-1559 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting1.png\" alt=\"self consistency prompting\" width=\"927\" height=\"279\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting1.png 927w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting1-300x90.png 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting1-768x231.png 768w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting1-18x5.png 18w\" sizes=\"(max-width: 927px) 100vw, 927px\" \/><\/p><p><img decoding=\"async\" class=\"alignnone wp-image-1560 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting2.png\" alt=\"self consistency prompting\" width=\"1021\" height=\"466\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting2.png 1021w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting2-300x137.png 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting2-768x351.png 768w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting2-18x8.png 18w\" sizes=\"(max-width: 1021px) 100vw, 1021px\" \/><\/p><p>En g\u00e9n\u00e9rant de nombreuses cha\u00eenes de pens\u00e9e et en prenant la r\u00e9ponse la plus courante (IMPORTANTE), nous pouvons obtenir une r\u00e9ponse correcte de mani\u00e8re plus coh\u00e9rente.<\/p><p>Il a \u00e9t\u00e9 d\u00e9montr\u00e9 que l\u2019autocoh\u00e9rence am\u00e9liore les r\u00e9sultats dans les t\u00e2ches d\u2019arithm\u00e9tique, de bon sens et de raisonnement symbolique. M\u00eame lorsque le CoT r\u00e9gulier s\u2019av\u00e9rait inefficace, l\u2019auto-coh\u00e9rence restait en mesure d\u2019am\u00e9liorer les r\u00e9sultats.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-066d2f9 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"066d2f9\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-bf6d1e4\" data-id=\"bf6d1e4\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-46d6498 elementor-widget elementor-widget-heading\" data-id=\"46d6498\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Code_pour_le_Self-Consistency\"><\/span>Code pour le Self-Consistency<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-68510ef elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"68510ef\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8af7bf6\" data-id=\"8af7bf6\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-501cecb elementor-widget elementor-widget-text-editor\" data-id=\"501cecb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Voici un r\u00e9cap du fonctionnement :<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1561 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting3.jpg\" alt=\"self consistency prompting\" width=\"474\" height=\"231\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting3.jpg 474w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting3-300x146.jpg 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting3-18x9.jpg 18w\" sizes=\"(max-width: 474px) 100vw, 474px\" \/><\/p><p>Pour utiliser l&rsquo;autocoh\u00e9rence, il est recommand\u00e9 d&rsquo;utiliser un script car les LLM avec interface ne permettent pas cette option. Vous trouverez le code Python pour utiliser l&rsquo;autocoh\u00e9rence :<\/p><pre>import logging\nfrom typing import List, Dict\nfrom difflib import SequenceMatcher\n\nlogger = logging.getLogger(__name__)\n\nclass AdvancedSelfConsistency:\n    def __init__(self, client, model: str,  num_samples: int = 5, similarity_threshold: float = 0.8):\n        self.client = client\n        self.model = model\n        self.num_samples = num_samples\n        self.similarity_threshold = similarity_threshold\n        self.self_consistency_completion_tokens = 0\n\n    def generate_responses(self, system_prompt: str, user_prompt: str) -&gt; List[str]:\n        responses = []\n        for _ in range(self.num_samples):\n            response = self.client.chat.completions.create(\n                model=self.model,\n                messages=[\n                    {\"role\": \"system\", \"content\": system_prompt},\n                    {\"role\": \"user\", \"content\": user_prompt}\n                ],\n                temperature=1,\n                max_tokens=4096\n            )\n            self.self_consistency_completion_tokens += response.usage.completion_tokens\n            responses.append(response.choices[0].message.content)\n        return responses\n\n    def calculate_similarity(self, a: str, b: str) -&gt; float:\n        return SequenceMatcher(None, a, b).ratio()\n\n    def cluster_similar_responses(self, responses: List[str]) -&gt; List[List[str]]:\n        clusters = []\n        for response in responses:\n            added_to_cluster = False\n            for cluster in clusters:\n                if self.calculate_similarity(response, cluster[0]) &gt;= self.similarity_threshold:\n                    cluster.append(response)\n                    added_to_cluster = True\n                    break\n            if not added_to_cluster:\n                clusters.append([response])\n        return clusters\n\n    def aggregate_results(self, responses: List[str]) -&gt; Dict[str, any]:\n        final_answers = responses\n        clusters = self.cluster_similar_responses(final_answers)\n        \n        cluster_info = []\n        for cluster in clusters:\n            cluster_info.append({\n                \"answer\": cluster[0],\n                \"frequency\": len(cluster),\n                \"variants\": cluster\n            })\n        \n        cluster_info.sort(key=lambda x: x['frequency'], reverse=True)\n        \n        return {\n            \"clusters\": cluster_info,\n            \"total_responses\": len(responses),\n            \"num_unique_clusters\": len(clusters)\n        }\n\n    def evaluate(self, system_prompt: str, user_prompt: str) -&gt; Dict[str, any]:\n        responses = self.generate_responses(system_prompt, user_prompt)\n        aggregated_result = self.aggregate_results(responses)\n        \n        return {\n            \"individual_responses\": responses,\n            \"aggregated_result\": aggregated_result\n        }\n\ndef advanced_self_consistency_approach(system_prompt: str, initial_query: str, client, model: str) -&gt; str:\n    self_consistency = AdvancedSelfConsistency(client, model)\n    result = self_consistency.evaluate(system_prompt, initial_query)\n    \n    logger.info(\"Advanced Self-Consistency Results:\")\n    logger.info(f\"Total responses: {result['aggregated_result']['total_responses']}\")\n    logger.info(f\"Number of unique clusters: {result['aggregated_result']['num_unique_clusters']}\")\n    for i, cluster in enumerate(result['aggregated_result']['clusters'], 1):\n        logger.debug(f\"\\nCluster {i}:\")\n        logger.debug(f\"  Representative answer: {cluster['answer']}\")\n        logger.debug(f\"  Frequency: {cluster['frequency']}\")\n        logger.debug(f\"  Variants: {cluster['variants']}\")\n    \n    if result['aggregated_result']['clusters']:\n        return result['aggregated_result']['clusters'][0]['answer'], self_consistency.self_consistency_completion_tokens\n    else:\n        return \"No consistent answer found.\", self_consistency.self_consistency_completion_tokens<\/pre>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-7a46550 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7a46550\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-323f444\" data-id=\"323f444\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-312a54b elementor-widget elementor-widget-heading\" data-id=\"312a54b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Self-consistency_sur_multiLLM\"><\/span>Self-consistency sur multiLLM<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-883215c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"883215c\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-746a494\" data-id=\"746a494\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-afecdc7 elementor-widget elementor-widget-text-editor\" data-id=\"afecdc7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\u00c9tant donn\u00e9 une cha\u00eene de r\u00e9flexion, plusieurs LLM peuvent \u00eatre utilis\u00e9s pour assurer la coh\u00e9rence entre les diff\u00e9rents LLM, am\u00e9liorant ainsi la pr\u00e9cision du r\u00e9sultat final. Initialement, la m\u00eame cha\u00eene de r\u00e9flexion est transmise \u00e0 plusieurs LLM diff\u00e9rents (GPT4, PaLM2, etc.). Ensuite, le quorum est d\u00e9termin\u00e9, dans cet exemple via GPT4 en tant qu&rsquo;\u00e9valuateur de quorum, mais peut \u00eatre programm\u00e9 \u00e0 l&rsquo;aide d&rsquo;autres m\u00e9thodologies.<\/p><p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1562 size-full\" src=\"http:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting4.png\" alt=\"self consistency prompting\" width=\"800\" height=\"308\" srcset=\"https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting4.png 800w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting4-300x116.png 300w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting4-768x296.png 768w, https:\/\/guillaume-guerard.com\/wp-content\/uploads\/2024\/10\/selfconsistencyprompting4-18x7.png 18w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/p><p>Le code est disponible sur ce <a href=\"https:\/\/github.com\/lastmile-ai\/aiconfig\/tree\/main\/cookbooks\/Multi-LLM-Consistency\" target=\"_blank\" rel=\"noopener\">GitHub.<\/a><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8e93aac elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8e93aac\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5ed3c2b\" data-id=\"5ed3c2b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-13d13fa elementor-widget elementor-widget-heading\" data-id=\"13d13fa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Self-consistency_meilleure_pratique\"><\/span>Self-consistency meilleure pratique<span class=\"ez-toc-section-end\"><\/span><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-fde77a0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fde77a0\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1a8df01\" data-id=\"1a8df01\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a63457e elementor-widget elementor-widget-text-editor\" data-id=\"a63457e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>La <a href=\"https:\/\/guillaume-guerard.com\/en\/cours-de-methodologie\/parfaire-la-methodologie\/\">m\u00e9thode<\/a> d&rsquo;auto-coh\u00e9rence comprend trois \u00e9tapes. Tout d&rsquo;abord, incitez un mod\u00e8le de langage \u00e0 l&rsquo;aide de l&rsquo;invite CoT, puis remplacez le \u00ab d\u00e9codage gourmand \u00bb (1-Best) dans l&rsquo;invite CoT par un \u00e9chantillonnage \u00e0 partir du d\u00e9codeur du mod\u00e8le de langage pour g\u00e9n\u00e9rer un ensemble diversifi\u00e9 de chemins de raisonnement, et enfin, marginalisez les chemins de raisonnement et agr\u00e9ger en choisissant la r\u00e9ponse la plus coh\u00e9rente dans l\u2019ensemble de r\u00e9ponses final.<\/p><p>Il est \u00e0 noter que l\u2019autocoh\u00e9rence peut \u00eatre harmonieusement int\u00e9gr\u00e9e \u00e0 la plupart des algorithmes d\u2019\u00e9chantillonnage, y compris, sans toutefois s\u2019y limiter, l\u2019\u00e9chantillonnage de temp\u00e9rature, l\u2019\u00e9chantillonnage top-k et l\u2019\u00e9chantillonnage de noyau.<\/p><p>N\u00e9anmoins, une telle op\u00e9ration peut n\u00e9cessiter l\u2019invocation de l\u2019API du mod\u00e8le pour affiner ces hyperparam\u00e8tres. \u00c0 la lumi\u00e8re de cela, une approche alternative pourrait consister \u00e0 permettre au mod\u00e8le de g\u00e9n\u00e9rer des r\u00e9sultats en utilisant diverses voies de raisonnement, puis \u00e0 g\u00e9n\u00e9rer un ensemble diversifi\u00e9 de voies de raisonnement candidates.<\/p><p>La r\u00e9ponse d\u00e9montrant le plus haut degr\u00e9 de coh\u00e9rence entre les diff\u00e9rentes trajectoires de raisonnement est alors plus encline \u00e0 repr\u00e9senter la solution exacte. L&rsquo;auto-coh\u00e9rence am\u00e9liore les r\u00e9sultats dans les t\u00e2ches de raisonnement arithm\u00e9tique, de bon sens et symbolique. De plus, en pratique, l\u2019autocoh\u00e9rence peut \u00eatre combin\u00e9e avec d\u2019autres techniques pour am\u00e9liorer encore les performances du mod\u00e8le. Il a \u00e9t\u00e9 constat\u00e9 que la combinaison de l\u2019autocoh\u00e9rence avec une approche de raisonnement en plusieurs \u00e9tapes guid\u00e9e par un discriminateur am\u00e9liorait consid\u00e9rablement les capacit\u00e9s de raisonnement du mod\u00e8le.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Home Data Science Self-Consistency, to consolidate the chain of thought Self-consistency is perhaps one of the most advanced techniques \u2026 <\/p>","protected":false},"author":1,"featured_media":0,"parent":47,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"elementor_header_footer","meta":{"footnotes":""},"class_list":["post-1553","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1553","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/comments?post=1553"}],"version-history":[{"count":4,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1553\/revisions"}],"predecessor-version":[{"id":1565,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/1553\/revisions\/1565"}],"up":[{"embeddable":true,"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/pages\/47"}],"wp:attachment":[{"href":"https:\/\/guillaume-guerard.com\/en\/wp-json\/wp\/v2\/media?parent=1553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}