<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Investigação Em Curso |</title><link>https://clementgodbarge.com/pt/tag/investigacao-em-curso/</link><atom:link href="https://clementgodbarge.com/pt/tag/investigacao-em-curso/index.xml" rel="self" type="application/rss+xml"/><description>Investigação Em Curso</description><generator>HugoBlox Kit (https://hugoblox.com)</generator><language>pt-pt</language><lastBuildDate>Sun, 12 Dec 2021 00:00:00 +0000</lastBuildDate><image><url>https://clementgodbarge.com/media/icon_hu_64a6c3bec25b3d7b.png</url><title>Investigação Em Curso</title><link>https://clementgodbarge.com/pt/tag/investigacao-em-curso/</link></image><item><title>The Spy Who Wished Them Well</title><link>https://clementgodbarge.com/pt/project/book1/</link><pubDate>Sun, 12 Dec 2021 00:00:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/project/book1/</guid><description>&lt;p&gt;Mais informações em breve.&lt;/p&gt;</description></item><item><title>Automatizar a marcação nas edições académicas digitais</title><link>https://clementgodbarge.com/pt/post/gpt3/</link><pubDate>Mon, 22 Nov 2021 18:15:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/post/gpt3/</guid><description>&lt;h1 id="introdução"&gt;Introdução&lt;/h1&gt;
&lt;p&gt;Como produzir edições académicas digitais sem arruinar o orçamento? Neste artigo, o primeiro de uma série dedicada à edição eficiente, avalio o papel que os modelos de linguagem pré-treinados podem desempenhar na automatização de tarefas editoriais como a marcação semântica.&lt;/p&gt;
&lt;details class="print:hidden xl:hidden" &gt;
&lt;summary&gt;Lista de Conteúdos&lt;/summary&gt;
&lt;div class="text-sm"&gt;
&lt;nav id="TableOfContents"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#uma-obra-de-amor"&gt;Uma obra de amor&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#uma-fasquia-alta"&gt;Uma fasquia alta&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#os-transformers-o-caminho-mais-simples-para-a-automatização"&gt;Os transformers: o caminho mais simples para a automatização?&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#codex"&gt;Codex&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#para-lá-da-openai"&gt;Para lá da OpenAI&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#experiência-1--categorização-de-textos"&gt;Experiência 1 – Categorização de textos.&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#engenharia-do-prompt"&gt;Engenharia do prompt&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#teste"&gt;Teste&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#resultado"&gt;Resultado&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#experiência-2--marcação-semântica"&gt;Experiência 2 – Marcação semântica&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#engenharia-do-prompt-1"&gt;Engenharia do prompt&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#teste-1"&gt;Teste&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/nav&gt;
&lt;/div&gt;
&lt;/details&gt;
&lt;h1 id="o-problema"&gt;O problema&lt;/h1&gt;
&lt;h2 id="uma-obra-de-amor"&gt;Uma obra de amor&lt;/h2&gt;
&lt;p&gt;Quem ama não conta os custos&amp;hellip; ou assim reza o velho ditado. Vale sobretudo para as edições académicas digitais: a transcrição, a tradução e a anotação que o seu desenvolvimento exige representam milhares de horas de trabalho, levadas a cabo, como no caso de
, por centenas de colaboradores altamente qualificados.&lt;/p&gt;
&lt;p&gt;Em certo sentido, é uma bênção que os projetos de grande visibilidade das humanidades digitais consigam reunir as somas avultadas de que precisam para funcionar. Mas depender tanto da generosidade de fundações abastadas, de universidades e de agências governamentais, e precisar durante tanto tempo de tantos recursos humanos, não constitui um modelo económico viável para o futuro.&lt;/p&gt;
&lt;p&gt;Com efeito, se queremos encorajar investigadores de todo o mundo a tornar documentos históricos acessíveis a um público mais vasto,
&lt;mark&gt;o custo das edições críticas digitais tem de baixar várias ordens de grandeza&lt;/mark&gt;
.&lt;/p&gt;
&lt;h2 id="uma-fasquia-alta"&gt;Uma fasquia alta&lt;/h2&gt;
&lt;p&gt;Um tanto paradoxalmente,
&lt;mark&gt;a solução talvez venha de projetos tão exigentes em mão-de-obra como
, porque constituem um valioso conjunto de treino&lt;/mark&gt;
para automatizar algumas das tarefas mais ingratas e repetitivas da edição digital, a começar pela marcação.&lt;/p&gt;
&lt;p&gt;Não que a marcação seja coisa de somenos. Pelo contrário:
&lt;mark&gt;a marcação tornou-se o componente indispensável de qualquer projeto académico digital que se leve a sério.&lt;/mark&gt;
Normalizada pela
, permite registar o maior número possível de aspetos do documento e do texto que este veicula: estrutura, anotações marginais, rasuras, variantes, tipo de papel, manchas, caligrafia&amp;hellip; o que se quiser.&lt;/p&gt;
&lt;p&gt;O exemplo seguinte, retirado de
, mostra como a marcação enriquece o texto com informação suplementar (categoria, estrutura, campos semânticos, rasuras, etc.), dando por fim às edições digitais uma vantagem considerável sobre as suas antepassadas em papel.&lt;/p&gt;
&lt;table&gt;
&lt;tr&gt;
&lt;th&gt; Texto simples &lt;/th&gt;
&lt;th&gt; Marcação XML&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Pour rompre grenades et donner
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;violence aux artifices de foeu
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Mects parmy la pouldre et la sixiesme
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;partye dicelle de vif argent
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/td&gt;
&lt;td&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;div&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;p008r_2&amp;#34;&lt;/span&gt; &lt;span class="na"&gt;categories=&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;arms and armor&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;head&amp;gt;&lt;/span&gt;Pour rompre &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;grenades&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; et donner&lt;span class="nt"&gt;&amp;lt;lb/&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;violence aux &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;artifices de foeu&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&amp;lt;/head&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;ab&amp;gt;&lt;/span&gt;Mects parmy la &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;pouldre&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;del&amp;gt;&amp;lt;ms&amp;gt;&lt;/span&gt;six fois autant&lt;span class="nt"&gt;&amp;lt;/ms&amp;gt;&lt;/span&gt; de
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;vif argent&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&amp;lt;/del&amp;gt;&amp;lt;lb/&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;del&amp;gt;&lt;/span&gt;et&lt;span class="nt"&gt;&amp;lt;/del&amp;gt;&lt;/span&gt; &lt;span class="nt"&gt;&amp;lt;ms&amp;gt;&lt;/span&gt;la sixiesme partye&lt;span class="nt"&gt;&amp;lt;/ms&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; dicelle de &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;vif argent&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&amp;lt;/ab&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;/div&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/table&gt;
&lt;p&gt;Esta informação não vale apenas para fins de arquivo: como já tive ocasião de mostrar, presta-se também à síntese e à análise. Ainda assim, este tipo de anotação pode consumir imenso tempo, tanto mais que o mesmo texto tem muitas vezes de existir em várias versões: tradução, transcrição, modernização, etc.&lt;/p&gt;
&lt;h1 id="a-solução"&gt;A solução&lt;/h1&gt;
&lt;h2 id="os-transformers-o-caminho-mais-simples-para-a-automatização"&gt;Os transformers: o caminho mais simples para a automatização?&lt;/h2&gt;
&lt;p&gt;Em 2020, a
lançou com grande alarido a sua mais recente família de modelos de linguagem de grande dimensão e uso geral, a que chamou GPT-3, sigla de «Generative Pre-trained Transformer 3». Os transformers representam um avanço bastante recente da inteligência artificial. Aprendem tarefas novas com uma rapidez impressionante, bastando-lhes ler uma instrução (prompt) e observar um número muito reduzido de exemplos. Podem ainda receber treino adicional com um conjunto de dados feito à medida (fine-tuning), o que melhora a latência e a precisão. Por isso se diz que o GPT-3 e os transformers comparáveis são
, aprendizes de poucos exemplos.&lt;/p&gt;
&lt;p&gt;A OpenAI afirma que o GPT-3 contém um número recorde de 175 mil milhões de parâmetros e que foi treinado com mais de 570 GB de texto, na maior parte documentos em inglês presumivelmente recolhidos
. Pela sua simples dimensão, o GPT-3 estabeleceu um novo padrão no domínio, executando de raiz as tarefas mais diversas com um realismo perturbador. Escreve
plausíveis,
em salas de chat,
,
, traduz documentos, explica jargão, e por aí fora.&lt;/p&gt;
&lt;p&gt;Com acesso antecipado à API da OpenAI desde maio de 2021, pude experimentar a capacidade do modelo para resolver várias tarefas de reputada dificuldade: traduzir poesia francesa e textos neolatinos para inglês, explicar analogias e até simplificar o livro 4 da &lt;em&gt;Fundamentação da Metafísica dos Costumes&lt;/em&gt; de Kant para uma criança de sete anos (sem grande convicção, diga-se).&lt;/p&gt;
&lt;h3 id="codex"&gt;Codex&lt;/h3&gt;
&lt;p&gt;Um dos desenvolvimentos mais recentes do GPT-3 incide sobre as linguagens de programação. Batizado &lt;em&gt;Codex&lt;/em&gt;, este modelo traduz linguagem natural em linguagem de programação e vice-versa. Se eu procurar, por exemplo, uma expressão regular que me permita «encontrar apenas as palavras que começam por maiúscula», o GPT-3 converte-a de imediato numa expressão regular funcional: &lt;code&gt;[A-Z]+\w+&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;A OpenAI afirma que o &lt;em&gt;Codex&lt;/em&gt; trabalha com uma dúzia de linguagens, entre as quais Python, JavaScript, Go, Perl, PHP, Ruby e Swift. Ao converter pseudocódigo em código sem esforço aparente, o &lt;em&gt;Codex&lt;/em&gt; permite concentrar a atenção não na sintaxe fastidiosa de uma linguagem, mas nos passos lógicos e nas estratégias que permitem a uma aplicação resolver problemas.&lt;/p&gt;
&lt;h3 id="para-lá-da-openai"&gt;Para lá da OpenAI&lt;/h3&gt;
&lt;p&gt;A OpenAI, claro está, não é a única a jogar. Como já referi, a Academia de Inteligência Artificial de Pequim anunciou em 2021 um modelo ainda maior e mais capaz, conhecido por &lt;em&gt;Wu Dao 2&lt;/em&gt;. A Nvidia e a Microsoft juntaram forças para produzir o modelo &lt;em&gt;Megatron-Turing NLG 530B&lt;/em&gt;, de nome bem escolhido. Start-ups mais pequenas, como a
e a
, também disponibilizam APIs ao público. Merecem ainda menção iniciativas de código aberto como a
. O panorama da IA evolui, de resto, muito depressa; para acompanhar as novas iniciativas do sector, consulte
.&lt;/p&gt;
&lt;h1 id="as-experiências"&gt;As experiências&lt;/h1&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;O objetivo destas experiências é encontrar o caminho mais económico para uma automatização fiável das tarefas editoriais. Poder-se-á objetar que algumas delas também se automatizariam com algoritmos de aprendizagem supervisionada. Exploraremos essa hipótese num próximo artigo.&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Poderá um transformer como o GPT-3 aprender a anotar, por exemplo, um manuscrito técnico e científico do século XVI?&lt;/p&gt;
&lt;h2 id="experiência-1--categorização-de-textos"&gt;Experiência 1 – Categorização de textos.&lt;/h2&gt;
&lt;p&gt;Comecemos por algo relativamente simples. Sendo um «few-shot learner», o GPT-3 deveria perceber depressa como a nossa equipa editorial classificou as entradas do Ms Fr 640.&lt;/p&gt;
&lt;h3 id="engenharia-do-prompt"&gt;Engenharia do prompt&lt;/h3&gt;
&lt;p&gt;Para o treinar, usei um prompt minimalista e escolhi como exemplos quatro entradas curtas em texto simples, entre as quais uma sobre «medicina», outra sobre «armas e armaduras» e outra sobre «pintura».&lt;/p&gt;
&lt;h3 id="teste"&gt;Teste&lt;/h3&gt;
&lt;p&gt;Copiei em seguida outra passagem, que não constava da sequência inicial:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Working neatly
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Never put down, if you can, two colors one on top of the other.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;But next, having made your design carefully, keep the place of shadows for them alone,
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&amp;amp; also separately that of lights &amp;amp; highlights, without layering one color all over &amp;amp; then highlighting or else shading on it.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;And in this way, you further your work, economize your colors &amp;amp; work neatly.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Which is the reason that, the colors not being muddled nor mixed together, they do not die &amp;amp; you soften the colors better, since they are not so thick.
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;O resultado é perfeitamente coerente com o conteúdo:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;categories&lt;/span&gt;&lt;span class="err"&gt;=&amp;#34;painting&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Se tentarmos com uma entrada pertencente a uma categoria que nem sequer figurava na seleção inicial de textos usada para treinar o GPT-3, o resultado surpreende.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;categories&lt;/span&gt;&lt;span class="err"&gt;=&amp;#34;jewelry&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="resultado"&gt;Resultado&lt;/h3&gt;
&lt;p&gt;A categoria «jewelry» (joalharia) não existe na nossa edição do Ms. Fr. 640: a equipa editorial
a categoria mais lata de «Stones» (pedras). A intuição do GPT-3 é, porém, boa, e indica que, com um pouco mais de treino, ele pode aprender a categorizar qualquer entrada do Ms. Fr. 640, e talvez até as de textos técnicos semelhantes do século XVI.&lt;/p&gt;
&lt;h2 id="experiência-2--marcação-semântica"&gt;Experiência 2 – Marcação semântica&lt;/h2&gt;
&lt;p&gt;Subamos um pouco a fasquia. Se transformers como o GPT-3 aprendem a categorizar textos segundo critérios editoriais específicos, conseguirão também identificar parte da marcação do texto?&lt;/p&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;&lt;em&gt;Secrets of Craft and Nature&lt;/em&gt;
etiquetas semânticas e estruturais. Infelizmente, o GPT-3 não processa imagens, ao contrário de outros projetos, como o
. É provável que futuras versões do GPT incluam essa capacidade, necessária para reconhecer a maior parte dos aspetos estruturais e materiais de um documento. Deixaremos de lado essas etiquetas em particular e concentrar-nos-emos na marcação que dispensa o reconhecimento de imagens.&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 id="engenharia-do-prompt-1"&gt;Engenharia do prompt&lt;/h3&gt;
&lt;p&gt;As etiquetas semânticas incluem referências a animais, plantas, topónimos, sensações, etc. No prompt de treino, selecionei alguns exemplos da edição:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Input prompt--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;The following is a list of words and their corresponding semantic tags
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cannons: &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;cannons&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;flasks: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;flasks&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;wooden: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;wooden&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;iron: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;iron&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;parchment: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;parchment&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;goats: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;goats&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;lambs: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;lambs&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;leather: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;leather&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;earth: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;earth&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;fine fatty earth: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;fine fatty earth&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Venice: &lt;span class="nt"&gt;&amp;lt;pl&amp;gt;&lt;/span&gt;Venice&lt;span class="nt"&gt;&amp;lt;/pl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Flemish: &lt;span class="nt"&gt;&amp;lt;pl&amp;gt;&lt;/span&gt;Flemish&lt;span class="nt"&gt;&amp;lt;/pl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;almond: &lt;span class="nt"&gt;&amp;lt;pa&amp;gt;&lt;/span&gt;almond&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;almond oil: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;almond&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; oil&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;walnuts skin: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;walnuts&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; skin&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;molten lead: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;molten lead&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;today: &lt;span class="nt"&gt;&amp;lt;tmp&amp;gt;&lt;/span&gt;today&lt;span class="nt"&gt;&amp;lt;/tmp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;In the past: &lt;span class="nt"&gt;&amp;lt;tmp&amp;gt;&lt;/span&gt;In the past&lt;span class="nt"&gt;&amp;lt;/tmp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Clockmakers: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;Clockmakers&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;red copper: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;red copper&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;crucible: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;crucible&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;bellows: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;bellows&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;charcoal: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;charcoal&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;founders: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;founders&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="teste-1"&gt;Teste&lt;/h3&gt;
&lt;p&gt;Experimentemos algumas palavras fáceis com o modelo &lt;code&gt;Davinci-codex&lt;/code&gt;: &lt;em&gt;Apothecary&lt;/em&gt;, &lt;em&gt;smoke&lt;/em&gt;, &lt;em&gt;glassmakers&lt;/em&gt;, &lt;em&gt;latten&lt;/em&gt; e &lt;em&gt;snake&lt;/em&gt;. Os resultados são imediatos e impecáveis:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Apothecary: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;Apothecary&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;smoke: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;smoke&lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;glassmakers: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;glassmakers&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;latten: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;latten&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;snake: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;snake&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Um teste mais difícil implica palavras compostas, como &lt;em&gt;copper plates&lt;/em&gt;, &lt;em&gt;walnut oil&lt;/em&gt; e &lt;em&gt;wood block&lt;/em&gt;. O que se pretende ver é se o GPT-3 lida bem com etiquetas aninhadas.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;copper plates: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;copper plates&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;walnut oil: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;walnut&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; oil&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;wood block: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;wood block&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Ora, os resultados são mistos: o &lt;code&gt;Davinci-codex&lt;/code&gt; só etiquetou corretamente &lt;em&gt;walnut oil&lt;/em&gt;, e não detetou as etiquetas aninhadas &lt;code&gt;tl&lt;/code&gt; e &lt;code&gt;m&lt;/code&gt; em &lt;em&gt;copper plates&lt;/em&gt; e &lt;em&gt;wood block&lt;/em&gt;. Como mostra o teste seguinte, porém, estes erros atenuam-se com um prompt de treino mais bem construído. Depois de acrescentar mais cinco exemplos de etiquetas aninhadas, o &lt;code&gt;Davinci-codex&lt;/code&gt; devolveu um resultado quase perfeito, com um único erro (&lt;em&gt;oil paintbrushes&lt;/em&gt;):&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cannon powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;wp&amp;gt;&lt;/span&gt;cannon&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;arquebus powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;wp&amp;gt;&lt;/span&gt;arquebus&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;oil paintbrushes: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;al&amp;gt;&lt;/span&gt;oil&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt; paintbrushes&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sheep footbones: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;al&amp;gt;&lt;/span&gt;sheep&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt; footbones&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;bronze mortar: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&amp;lt;m&amp;gt;&lt;/span&gt;bronze&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt; mortar&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id="conclusão"&gt;Conclusão&lt;/h1&gt;
&lt;p&gt;Convém não esquecer que estes testes foram feitos com pequenos fragmentos de texto. Suspeito que, com mais contexto nos exemplos e no prompt, os modelos GPT-3 dariam resultados ainda melhores. De resto, afinar o modelo (fine-tuning) com conjuntos de dados de treino feitos à medida melhoraria sem dúvida a precisão da etiquetagem.&lt;br&gt;
Embora estas experiências ainda tenham de ser conduzidas em maior escala para demonstrar a fiabilidade dos modelos de linguagem pré-treinados, podemos apesar de tudo concluir que
&lt;mark&gt;esta abordagem permite aos editores automatizar várias tarefas de anotação em poucos passos simples, com uma poupança potencial enorme de tempo e de dinheiro.&lt;/mark&gt;
&lt;/p&gt;</description></item><item><title>Visualizar manuscritos 2 (atualização)</title><link>https://clementgodbarge.com/pt/post/treemap2/</link><pubDate>Sat, 20 Nov 2021 16:00:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/post/treemap2/</guid><description>&lt;p&gt;Como prometido, eis uma nova versão do treemap interativo apresentado num
, desta vez com dois modos de visualização.&lt;/p&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;Para uma melhor experiência de visualização, certifique-se de que a página está em modo claro (clique no ícone da lua, no canto superior direito).&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;head&gt;
&lt;meta charset="UTF-8" /&gt;
&lt;meta http-equiv="X-UA-Compatible" content="IE=edge" /&gt;
&lt;meta name="viewport" content="width=device-width, initial-scale=1.0" /&gt;
&lt;title&gt;&lt;/title&gt;
&lt;link rel="preconnect" href="https://fonts.gstatic.com" /&gt;
&lt;link
href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;700&amp;display=swap"
rel="stylesheet"
/&gt;
&lt;link rel="stylesheet" href="css/index.css" /&gt;
&lt;link rel="stylesheet" href="css/vis-treemap.css" /&gt;
&lt;link rel="stylesheet" href="css/vis-tooltip.css" /&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;div class="stacked"&gt;
&lt;div class="switch"&gt;
&lt;input
type="checkbox"
name="group-by-category-switch"
id="group-by-category-switch"
checked
/&gt;
&lt;label for="group-by-category-switch"&gt; Group folios by category &lt;/label&gt;
&lt;/div&gt;
&lt;div id="treemap"&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;script src="https://cdn.jsdelivr.net/npm/d3@7.9.0/dist/d3.min.js" integrity="sha384-CjloA8y00+1SDAUkjs099PVfnY2KmDC2BZnws9kh8D/lX1s46w6EPhpXdqMfjK6i" crossorigin="anonymous" referrerpolicy="no-referrer"&gt;&lt;/script&gt;
&lt;script src="js/vis-treemap.js"&gt;&lt;/script&gt;
&lt;script src="js/vis-tooltip.js"&gt;&lt;/script&gt;
&lt;script src="js/index.js"&gt;&lt;/script&gt;
&lt;/body&gt;</description></item><item><title>O arquivo num relance</title><link>https://clementgodbarge.com/pt/post/dashboard/</link><pubDate>Mon, 24 May 2021 16:00:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/post/dashboard/</guid><description>&lt;h1 id="o-problema"&gt;O problema&lt;/h1&gt;
&lt;p&gt;Os arquivos históricos podem ser de uma desordem intimidante. O &lt;em&gt;Mediceo del Principato&lt;/em&gt;, no
, é um caso exemplar. Com efeito, só uma pequena parte está inventariada, e muitos dos seus documentos andam dispersos por mais de 6500 volumes sem razão aparente. Para complicar as coisas, o arquivo só permite consultar um número limitado de volumes – ou &lt;em&gt;filze&lt;/em&gt;, como lá lhes chamam: maços, diríamos nós. Em tempos normais, o limite é de 4 &lt;em&gt;filze&lt;/em&gt; por dia; em tempo de pandemia, porém, desceu para 4 de duas em duas semanas. Na falta de inventários detalhados, a dimensão considerável do arquivo obriga os investigadores a engendrar estratégias para encontrar depressa os documentos que procuram.&lt;/p&gt;
&lt;h1 id="a-solução"&gt;A solução&lt;/h1&gt;
&lt;p&gt;Uns preferem confiar no acaso; outros tentam também fazer conjeturas informadas a partir da cronologia, dos destinatários, dos autores, da origem do fundo de arquivo, da língua, etc. &lt;em&gt;Olhar&lt;/em&gt; para todas estas variáveis ao mesmo tempo, porém, pode revelar padrões inesperados na estrutura do arquivo e melhorar as nossas conjeturas. A minha experiência mostra que, uma vez representados graficamente, os metadados que os investigadores costumam reunir numa folha de cálculo aumentam significativamente a consciência situacional no arquivo.&lt;/p&gt;
&lt;h1 id="a-experiência"&gt;A experiência&lt;/h1&gt;
&lt;p&gt;A minha investigação atual centra-se na correspondência de um espião do século XVI. As suas cartas estão espalhadas por centenas de &lt;em&gt;filze&lt;/em&gt;. Foram escritas sob identidades diferentes, a destinatários diferentes e por vezes inesperados, de lugares diferentes, etc. Para encontrar as &lt;em&gt;filze&lt;/em&gt; com mais probabilidade de conter as cartas esperadas, montei um dashboard, uma aplicação web de visualização interativa de dados (
) que cruza informação de toda a espécie, geográfica e cronológica incluída, com um diagrama hierárquico (
) do fundo de arquivo. O dashboard diz-me num relance o que já foi encontrado, quanto isso representa, e dá-me uma ideia aproximada de onde poderia procurar cartas novas. Além disso, ao clicar em variáveis específicas, todos os diagramas se atualizam para mostrar correlações particulares.&lt;/p&gt;
&lt;h1 id="próximos-passos"&gt;Próximos passos&lt;/h1&gt;
&lt;p&gt;Mais importante ainda, talvez: este dashboard pode ser reconvertido em índice visual. Quando a edição crítica destas cartas for publicada em linha, o dashboard funcionará como porta de entrada alternativa, a partir da qual os leitores poderão percorrer os dados. Por razões de confidencialidade, só posso mostrar por agora uma captura de ecrã com partes ocultadas, mas divulgarei o dashboard completo no próximo ano. Entretanto, um protótipo estará disponível em breve. Fiquem atentos!&lt;/p&gt;</description></item><item><title>Visualizar a marcação semântica do BnF Ms. Fr. 640</title><link>https://clementgodbarge.com/pt/post/visualization/</link><pubDate>Sun, 20 Dec 2020 18:15:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/post/visualization/</guid><description>&lt;h1 id="panorâmica"&gt;Panorâmica&lt;/h1&gt;
&lt;p&gt;As edições académicas ricas em dados contêm anotações editoriais valiosas que se podem extrair, analisar e visualizar para toda a espécie de fins académicos. É o caso de
, lançada em 2020, que disponibiliza o seu ficheiro de metadados para descarga no seu repositório GitHub. Neste artigo, mostro como reunir todas estas variáveis numa matriz de correlação e visualizá-las de diferentes maneiras.&lt;/p&gt;
&lt;h1 id="os-dados"&gt;Os dados&lt;/h1&gt;
&lt;p&gt;O Making and Knowing Project gera uma folha de cálculo com informação atualizada sobre o conteúdo do manuscrito: &lt;code&gt;entry_metadata.csv&lt;/code&gt;. O ficheiro pode ser obtido no
do Making &amp;amp; Knowing. Em alternativa, podem gerar-se ficheiros .csv à medida, com mais marcação, graças ao excelente
de Matthew Kumar, uma versão em Python do BnF Ms. Fr. 640.&lt;/p&gt;
&lt;h2 id="preparar-o-python"&gt;Preparar o Python&lt;/h2&gt;
&lt;p&gt;Usaremos o Pandas para tratar os dados, o Matplotlib e o seaborn para os mapas de calor e, por fim, o NetworkX para produzir redes baseadas em correlações.&lt;br&gt;
Para este tipo de variáveis, evitaremos o método de Pearson e usaremos em seu lugar o método 𝜙𝐾. Convém
este método de correlação e sobre o &lt;code&gt;PhiK&lt;/code&gt;, a biblioteca correspondente.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;#install packages&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="n"&gt;phik&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# import modules&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;pandas&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;pd&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;plt&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;seaborn&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;sns&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;networkx&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;nx&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id="preparar-os-dados"&gt;Preparar os dados&lt;/h2&gt;
&lt;p&gt;Comecemos por descarregar o ficheiro de metadados mais recente da edição, na pasta metadata do seu
.
Selecionaremos apenas as colunas de que precisamos. Para esta demonstração, escolho todas as etiquetas semânticas da tradução inglesa &lt;code&gt;tl&lt;/code&gt;, mas também se podem escolher as da transcrição francesa &lt;code&gt;tc&lt;/code&gt; ou as da versão normalizada &lt;code&gt;tcn&lt;/code&gt;.
Os dados vêm em valores separados por ponto e vírgula, e precisamos que o Python os conte por nós. Usaremos para isso o método stack-unstack com a expressão regular &lt;code&gt;[^;\s][^\;]*[^;\s]*&lt;/code&gt;.
Para tornar a matriz mais legível, mudamos o nome a cada coluna. Quem tiver pressa pode saltar este passo; basta ter presente que, nesta altura, o nosso dataframe se chama &lt;code&gt;tagsrn&lt;/code&gt;.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# load the edition&amp;#39;s metadata&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;df&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;read_csv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;entry_metadata.csv&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# select the tags you want to correlate&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;dftags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;df&lt;/span&gt;&lt;span class="p"&gt;[[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;al_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;bp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;cn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;df_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;env_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;m_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;md_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;ms_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;mu_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pa_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pro_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tmp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;wp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;de_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;el_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;it_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;la_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;oc_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;po_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# count comma separated values&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;tagcount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;dftags&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dropna&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;str&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;[^;\s][^\;]*[^;\s]*&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;unstack&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# rename columns&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;tagsrn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tagcount&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rename&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;columns&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;al_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;animals&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;bp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;body parts&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;cn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;currency&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;df_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;definitions&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;env_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;environment&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;m_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;material&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;md_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;medical&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;ms_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;measurement&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;mu_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;music&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pa_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;plant&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;toponym&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;person&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pro_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;profession&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sensory&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tool&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tmp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;temporal&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;wp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;weapons&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;de_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;German&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;el_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Greek&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;it_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Italian&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;la_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Italian&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;oc_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Occitan&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;po_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Poitevin&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id="correlacionar"&gt;Correlacionar&lt;/h1&gt;
&lt;p&gt;Limpo o dataframe, podemos passar ao cálculo dos coeficientes de correlação entre cada par de variáveis. Nesta fase, é importante compreender os dados e garantir que se usa o método de correlação mais adequado. O pacote &lt;code&gt;pandas-profiling&lt;/code&gt; é particularmente útil para isso.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# calculate correlation coefficient with the phi k method&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;cortag&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tagsrn&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;phik_matrix&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;cortag&lt;/code&gt; é a nossa matriz de correlação. Podemos agora experimentar diferentes tipos de visualização.&lt;/p&gt;
&lt;h1 id="visualizar"&gt;Visualizar&lt;/h1&gt;
&lt;p&gt;A primeira coisa a tentar é visualizá-la como matriz codificada por cores, com o
do &lt;code&gt;seaborn&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id="mapa-de-calor-de-correlações"&gt;Mapa de calor de correlações&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ax&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;subplots&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;figsize&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;ax&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sns&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;heatmap&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;linewidths&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;.03&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vmin&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cmap&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Oranges&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;square&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Mapa de calor das correlações do BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/heatmap_hu_943abdd98e24b0a3.webp 320w, https://clementgodbarge.com/post/visualization/heatmap_hu_eca41311a0e0ac17.webp 480w, https://clementgodbarge.com/post/visualization/heatmap_hu_1650defb643c0285.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/heatmap_hu_943abdd98e24b0a3.webp"
width="760"
height="665"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Quem conhece bem o texto vê logo que o mapa de calor faz muito sentido. Os nomes, por exemplo, estão fortemente correlacionados com o latim, pois era costume, sobretudo entre os humanistas do século XVI, latinizá-los.&lt;/p&gt;
&lt;p&gt;Haverá quem diga que este mapa de calor se limita a afirmar o óbvio. Não estão inteiramente errados e, à primeira vista, as etiquetas médicas parecem confirmá-lo, pois correlacionam-se, como seria de prever, com partes do corpo, medidas e plantas.&lt;/p&gt;
&lt;p&gt;Mas, se lermos o mapa de calor com mais atenção, linha a linha, podemos encontrar correlações interessantes e inesperadas. Que as etiquetas médicas, por exemplo, se correlacionem com palavras italianas e latinas dá-nos pistas sobre a origem das receitas médicas do Ms. Fr. 640. Do mesmo modo, a correlação entre profissões, definições e medidas mostra até que ponto a identidade profissional estrutura os discursos técnicos do século XVI.&lt;/p&gt;
&lt;h3 id="mapa-de-agrupamentos-clustermap"&gt;Mapa de agrupamentos (clustermap)&lt;/h3&gt;
&lt;p&gt;Os mapas de calor são úteis em contextos «exploratórios», mas podem parecer um pouco confusos ao público, sobretudo quando se discutem – ou ainda se procuram – agrupamentos semânticos específicos no manuscrito. O módulo &lt;code&gt;clustermap&lt;/code&gt; do seaborn pode dar resultados interessantes.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;clustermap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sns&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clustermap&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;figsize&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;13&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;dendrogram_ratio&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;.1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.2&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;vmin&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cmap&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Oranges&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cbar_pos&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;.06&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.03&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.68&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Mapa de agrupamentos das correlações do BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/clustermap_hu_677e6e5028f9d902.webp 320w, https://clementgodbarge.com/post/visualization/clustermap_hu_2dac945fa40a8ac6.webp 480w, https://clementgodbarge.com/post/visualization/clustermap_hu_3abaf7b36a53572.webp 753w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/clustermap_hu_677e6e5028f9d902.webp"
width="753"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Além de parecer um inseto pixelizado (sim, a palavra está dicionarizada), o clustermap distingue claramente as etiquetas isoladas (em cima e à esquerda) das mais interligadas. Distinguem-se também agrupamentos isolados, como música e poitevin (quem diria!), de outros mais centrais, como medidas, material, definições e armas. As profissões estão mais interligadas, mas não fazem parte, pelo menos nesta matriz de correlação, de nenhum agrupamento em particular.&lt;/p&gt;
&lt;h3 id="rede-de-correlações"&gt;Rede de correlações&lt;/h3&gt;
&lt;p&gt;Se quisermos sintetizar ainda mais as correlações contidas na matriz, os grafos de rede oferecem uma solução elegante. Isto é particularmente verdade nos contextos em que queremos comunicar sobre o conteúdo do manuscrito.&lt;br&gt;
Para o fazer, é preciso transformar a matriz numa lista de arestas e nós e definir um limiar que elimine do grafo as correlações mais fracas.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# transform the data&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stack&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reset_index&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;columns&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# threshold &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;loc&lt;/span&gt;&lt;span class="p"&gt;[(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;.6&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;])]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# create edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;G&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;from_pandas_edgelist&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# draw network using Kamada &amp;amp; Kawai&amp;#39;s algorithm &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;figure&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="n"&gt;figsize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;draw_kamada_kawai&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;G&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;with_labels&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node_color&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;red&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;edge_color&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;black&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;linewidths&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;font_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Grafo das correlações do BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/graph_hu_396032c00cb7f7f3.webp 320w, https://clementgodbarge.com/post/visualization/graph_hu_85295e5fba6293bc.webp 480w, https://clementgodbarge.com/post/visualization/graph_hu_4190dbca7e86ffc1.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/graph_hu_396032c00cb7f7f3.webp"
width="760"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Se houver arestas e nós a mais, pode sempre alterar-se o limiar para obter um resultado mais limpo. Em alternativa, pode exportar-se o grafo para o manipular no &lt;code&gt;Gephi&lt;/code&gt;, com a função &lt;code&gt;.write_gexf()&lt;/code&gt;.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;write_gexf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;G&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;graph.gexf&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;O resultado pode ver-se no início deste artigo.&lt;/p&gt;
&lt;h3 id="atualização-rede-ponderada-circular"&gt;Atualização: rede ponderada circular&lt;/h3&gt;
&lt;p&gt;Andava à procura de maneiras de representar matrizes de correlação como redes ponderadas e encontrei esta abordagem interessante
, que adapto aqui ao nosso conjunto de dados.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# create graph weighted by correlation coefficients (unfiltered)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Gx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;from_pandas_edgelist&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;edge_attr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# determine a threshold to remove some edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;threshold&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;0.4&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# list to store edges to remove&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;remove&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# loop through edges in Gx and find correlations which are below the threshold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;var2&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;edges&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;corr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;var2&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="c1"&gt;#add to remove node list if abs(corr) &amp;lt; threshold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;corr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;threshold&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;append&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;var2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# remove edges contained in the remove list&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;remove_edges_from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39; edges removed&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Removidas algumas arestas, podemos determinar a cor e a espessura das restantes.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# determine the colors of edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_colour&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;correlation&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mf"&gt;0.8&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;#ff872c&amp;#39;&lt;/span&gt; &lt;span class="c1"&gt;# orange&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;#f11d28&amp;#39;&lt;/span&gt; &lt;span class="c1"&gt;# red&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_thickness&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;benchmark_thickness&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;benchmark_thickness&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;scaling_factor&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_node_size&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;degree&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;degree&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Damos também aos nós um tamanho proporcional ao seu número de ligações.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# assign node size depending on number of connections (degree)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;node_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;degree&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;items&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;node_size&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;assign_node_size&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;O resultado é um grafo ponderado que admite mais nós e consideravelmente mais arestas, sem deixar de ser legível e informativo.&lt;/p&gt;
&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Grafo ponderado das correlações do BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/weightedgraph_hu_7b75f2e5fc7edb1b.webp 320w, https://clementgodbarge.com/post/visualization/weightedgraph_hu_9f0550eb49c17f11.webp 480w, https://clementgodbarge.com/post/visualization/weightedgraph_hu_a9106989817361fb.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/weightedgraph_hu_7b75f2e5fc7edb1b.webp"
width="760"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;</description></item><item><title>A correspondência secreta de Filippo Cavriana, 1568—1589.</title><link>https://clementgodbarge.com/pt/project/cavrianas-correspondence/</link><pubDate>Fri, 27 Dec 2019 00:00:00 +0000</pubDate><guid>https://clementgodbarge.com/pt/project/cavrianas-correspondence/</guid><description>&lt;p&gt;As cartas secretas de Filippo Cavriana são um testemunho único das guerras religiosas francesas: cobrem, ao longo de mais de 20 anos, as intrigas, batalhas, negociações e conspirações que marcaram o ritmo da vida na corte de França e fora dela.&lt;/p&gt;
&lt;p&gt;As cartas, conservadas no
, estão na sua maior parte dispersas por centenas de maços não inventariados – &lt;em&gt;filze&lt;/em&gt;, como lhes chamam em Florença. Embora tenha conseguido descobrir dezenas de cartas novas nos últimos anos, este tipo de investigação em arquivo revela-se cada vez mais difícil. A distância geográfica de Florença, a degradação de alguns serviços públicos em Itália e a pandemia de Covid-19 concorrem para fazer destes projetos de publicação um processo longo e incerto.&lt;/p&gt;
&lt;p&gt;Publicar fontes primárias deve continuar a fazer parte, apesar das dificuldades crescentes, do ofício do historiador. A forma dessas publicações, porém, terá provavelmente de evoluir: adaptar-se à situação que acabo de descrever, sem dúvida, mas também refletir melhor a natureza incremental desta investigação e permitir que outra pessoa a continue.&lt;/p&gt;
&lt;p&gt;Daí a ideia de desenvolver um novo tipo de projeto de publicação digital que seja &lt;strong&gt;escalável&lt;/strong&gt;, &lt;strong&gt;sustentável&lt;/strong&gt;, &lt;strong&gt;fiável&lt;/strong&gt;, &lt;strong&gt;independente de plataforma&lt;/strong&gt; e potencialmente &lt;strong&gt;colaborativo&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;A edição está agora em linha: &lt;strong&gt;
&lt;/strong&gt;. Reúne as cartas transcritas até à data, dos Arquivos de Estado de Florença e de Mântua e da BnF, e cresce à medida que surgem outras. Expliquei as razões do seu desenho num
.&lt;/p&gt;</description></item></channel></rss>