<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Ricerca in Corso |</title><link>https://clementgodbarge.com/it/tag/ricerca-in-corso/</link><atom:link href="https://clementgodbarge.com/it/tag/ricerca-in-corso/index.xml" rel="self" type="application/rss+xml"/><description>Ricerca in Corso</description><generator>HugoBlox Kit (https://hugoblox.com)</generator><language>it-it</language><lastBuildDate>Sun, 12 Dec 2021 00:00:00 +0000</lastBuildDate><image><url>https://clementgodbarge.com/media/icon_hu_64a6c3bec25b3d7b.png</url><title>Ricerca in Corso</title><link>https://clementgodbarge.com/it/tag/ricerca-in-corso/</link></image><item><title>The Spy Who Wished Them Well</title><link>https://clementgodbarge.com/it/project/book1/</link><pubDate>Sun, 12 Dec 2021 00:00:00 +0000</pubDate><guid>https://clementgodbarge.com/it/project/book1/</guid><description>&lt;p&gt;Prossimamente maggiori dettagli.&lt;/p&gt;</description></item><item><title>Automatizzare la marcatura nelle edizioni critiche digitali</title><link>https://clementgodbarge.com/it/post/gpt3/</link><pubDate>Mon, 22 Nov 2021 18:15:00 +0000</pubDate><guid>https://clementgodbarge.com/it/post/gpt3/</guid><description>&lt;h1 id="introduzione"&gt;Introduzione&lt;/h1&gt;
&lt;p&gt;Come si producono edizioni critiche digitali senza svenarsi? Con questo post apro una serie dedicata all’edizione efficiente; qui valuto quale parte possano avere i modelli linguistici pre-addestrati nell’automazione delle operazioni editoriali, a cominciare dalla marcatura semantica.&lt;/p&gt;
&lt;details class="print:hidden xl:hidden" &gt;
&lt;summary&gt;Indice dei Contenuti&lt;/summary&gt;
&lt;div class="text-sm"&gt;
&lt;nav id="TableOfContents"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#un-lavoro-damore"&gt;Un lavoro d’amore&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#una-soglia-alta"&gt;Una soglia alta&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#i-transformer-la-via-più-breve-allautomazione"&gt;I transformer: la via più breve all’automazione?&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#codex"&gt;Codex&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#oltre-openai"&gt;Oltre OpenAI&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#esperimento-1--categorizzazione-dei-testi"&gt;Esperimento 1 – Categorizzazione dei testi&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#prompt-engineering"&gt;Prompt engineering&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#verifica"&gt;Verifica&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#risultato"&gt;Risultato&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#esperimento-2--marcatura-semantica"&gt;Esperimento 2 – Marcatura semantica&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#prompt-engineering-1"&gt;Prompt engineering&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#verifica-1"&gt;Verifica&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/nav&gt;
&lt;/div&gt;
&lt;/details&gt;
&lt;h1 id="il-problema"&gt;Il problema&lt;/h1&gt;
&lt;h2 id="un-lavoro-damore"&gt;Un lavoro d’amore&lt;/h2&gt;
&lt;p&gt;Quando c’è di mezzo l’amore non si bada a spese… o così, almeno, vuole il proverbio. E vale a maggior ragione per le edizioni critiche digitali: trascrivere, tradurre e annotare significa migliaia di ore di lavoro, che in un caso come quello di
sono state prestate da centinaia di collaboratori di altissima qualificazione.&lt;/p&gt;
&lt;p&gt;Per certi versi è una fortuna che i progetti più in vista dell’umanistica digitale riescano a raccogliere le somme enormi di cui hanno bisogno. Ma una dipendenza così stretta dalla munificenza di fondazioni facoltose, università e agenzie governative, unita al bisogno prolungato di tante risorse umane, non è un modello economico su cui costruire il futuro.&lt;/p&gt;
&lt;p&gt;Anzi: se vogliamo che studiosi di ogni parte del mondo rendano i documenti storici accessibili a un pubblico più vasto,
&lt;mark&gt;il costo delle edizioni critiche digitali dovrebbe scendere di parecchi ordini di grandezza&lt;/mark&gt;
.&lt;/p&gt;
&lt;h2 id="una-soglia-alta"&gt;Una soglia alta&lt;/h2&gt;
&lt;p&gt;Paradossalmente,
&lt;mark&gt;la soluzione potrebbe venire proprio dai progetti più affamati di manodopera, come
, che costituiscono un prezioso insieme di addestramento&lt;/mark&gt;
per automatizzare le operazioni più ingrate e ripetitive dell’edizione digitale, a cominciare dalla marcatura.&lt;/p&gt;
&lt;p&gt;Non che la marcatura sia cosa da poco. Al contrario:
&lt;mark&gt;la marcatura è ormai la componente irrinunciabile di ogni progetto digitale che si rispetti.&lt;/mark&gt;
Normalizzata dalla
, permette di registrare del documento, e del testo che esso trasmette, quanti più aspetti si vogliano: struttura, note marginali, cancellature, varianti, tipo di carta, macchie, grafia… tutto quello che vi viene in mente.&lt;/p&gt;
&lt;p&gt;L’esempio che segue, tratto da
, mostra come la marcatura arricchisca il testo di informazioni ulteriori (categoria, struttura, campi semantici, cancellature e via dicendo), dando in definitiva alle edizioni digitali un vantaggio netto sui loro antenati di carta.&lt;/p&gt;
&lt;table&gt;
&lt;tr&gt;
&lt;th&gt; Testo semplice &lt;/th&gt;
&lt;th&gt; Marcatura XML&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Pour rompre grenades et donner
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;violence aux artifices de foeu
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Mects parmy la pouldre et la sixiesme
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;partye dicelle de vif argent
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/td&gt;
&lt;td&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;div&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;p008r_2&amp;#34;&lt;/span&gt; &lt;span class="na"&gt;categories=&lt;/span&gt;&lt;span class="s"&gt;&amp;#34;arms and armor&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;head&amp;gt;&lt;/span&gt;Pour rompre &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;grenades&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; et donner&lt;span class="nt"&gt;&amp;lt;lb/&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;violence aux &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;artifices de foeu&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&amp;lt;/head&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;ab&amp;gt;&lt;/span&gt;Mects parmy la &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;pouldre&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;del&amp;gt;&amp;lt;ms&amp;gt;&lt;/span&gt;six fois autant&lt;span class="nt"&gt;&amp;lt;/ms&amp;gt;&lt;/span&gt; de
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;vif argent&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&amp;lt;/del&amp;gt;&amp;lt;lb/&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;del&amp;gt;&lt;/span&gt;et&lt;span class="nt"&gt;&amp;lt;/del&amp;gt;&lt;/span&gt; &lt;span class="nt"&gt;&amp;lt;ms&amp;gt;&lt;/span&gt;la sixiesme partye&lt;span class="nt"&gt;&amp;lt;/ms&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; dicelle de &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;vif argent&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&amp;lt;/ab&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;/div&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/table&gt;
&lt;p&gt;Informazioni preziose non soltanto a fini di conservazione, ma anche – come ho avuto modo di mostrare altrove – per la sintesi e l’analisi. Il guaio è che annotare in questo modo costa un tempo enorme, tanto più che lo stesso testo deve spesso esistere in più vesti: come traduzione, come trascrizione, come versione modernizzata e così via.&lt;/p&gt;
&lt;h1 id="la-soluzione"&gt;La soluzione&lt;/h1&gt;
&lt;h2 id="i-transformer-la-via-più-breve-allautomazione"&gt;I transformer: la via più breve all’automazione?&lt;/h2&gt;
&lt;p&gt;Nel 2020
ha presentato in pompa magna la sua nuova famiglia di modelli linguistici generalisti su larga scala, GPT-3, sigla di «Generative Pre-trained Transformer 3». I transformer sono una svolta recente dell’intelligenza artificiale: imparano compiti nuovi con una rapidità sconcertante, semplicemente leggendo un prompt e guardando un pugno di esempi; e possono ricevere un addestramento supplementare su un insieme di dati ad hoc (il cosiddetto fine-tuning), che ne migliora latenza e precisione. Per questo si dice che GPT-3 e i transformer analoghi sono
.&lt;/p&gt;
&lt;p&gt;OpenAI dichiara per GPT-3 la cifra record di 175 miliardi di parametri e un addestramento su oltre 570 GB di testo, in massima parte documenti in inglese presumibilmente raccolti da
. Per le sole dimensioni, GPT-3 ha fissato un nuovo metro di paragone nel settore: esegue fin da subito i compiti più diversi con un realismo che mette a disagio. Scrive
plausibili,
nelle chat,
,
, traduce documenti, spiega il gergo, e via dicendo.&lt;/p&gt;
&lt;p&gt;Avendo accesso in anteprima all’API di OpenAI dal maggio 2021, ho potuto mettere alla prova il modello su una serie di compiti che passano per difficili: tradurre in inglese poesia francese e testi neolatini, spiegare analogie, perfino semplificare il quarto libro della &lt;em&gt;Fondazione della metafisica dei costumi&lt;/em&gt; di Kant per un bambino di sette anni (con esiti, va detto, poco convincenti).&lt;/p&gt;
&lt;h3 id="codex"&gt;Codex&lt;/h3&gt;
&lt;p&gt;Uno degli ultimi sviluppi di GPT-3 riguarda i linguaggi di programmazione. Il modello, battezzato &lt;em&gt;Codex&lt;/em&gt;, traduce il linguaggio naturale in linguaggio informatico e viceversa. Se cerco, poniamo, un’espressione regolare che «trovi soltanto le parole che iniziano con la maiuscola», GPT-3 me la restituisce subito, e funzionante: &lt;code&gt;[A-Z]+\w+&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Secondo OpenAI, &lt;em&gt;Codex&lt;/em&gt; sa lavorare con una dozzina di linguaggi, fra cui Python, JavaScript, Go, Perl, PHP, Ruby e Swift. Convertendo senza attriti lo pseudocodice in codice, permette di concentrarsi non sulla sintassi pignola di un linguaggio, ma sui passaggi logici e sulle strategie con cui un’applicazione risolve un problema.&lt;/p&gt;
&lt;h3 id="oltre-openai"&gt;Oltre OpenAI&lt;/h3&gt;
&lt;p&gt;OpenAI, si capisce, non è sola in campo. Come si è detto, nel 2021 la Beijing Academy of Artificial Intelligence ha annunciato un modello ancora più grande e più capace, &lt;em&gt;Wu Dao 2&lt;/em&gt;. Nvidia e Microsoft hanno unito le forze per produrre &lt;em&gt;Megatron-Turing NLG 530B&lt;/em&gt;, un nome che dice tutto. Start-up più piccole come
e
offrono anch’esse API al pubblico, e meritano una menzione le iniziative open source come
. Il panorama, si sa, cambia in fretta: per seguire le novità del settore, tenete d’occhio
.&lt;/p&gt;
&lt;h1 id="gli-esperimenti"&gt;Gli esperimenti&lt;/h1&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;L’obiettivo di questi esperimenti è individuare la via più economica verso un’automazione affidabile delle operazioni editoriali. Si potrebbe obiettare che alcune di esse si prestano anche ad algoritmi di apprendimento supervisionato: è un’ipotesi che esploreremo in un prossimo post.&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Un transformer come GPT-3 può imparare ad annotare, poniamo, un manoscritto tecnico-scientifico del Cinquecento?&lt;/p&gt;
&lt;h2 id="esperimento-1--categorizzazione-dei-testi"&gt;Esperimento 1 – Categorizzazione dei testi&lt;/h2&gt;
&lt;p&gt;Partiamo da qualcosa di relativamente semplice. Da bravo «few-shot learner», GPT-3 dovrebbe capire in fretta secondo quali criteri il nostro gruppo di lavoro ha classificato le voci del Ms. Fr. 640.&lt;/p&gt;
&lt;h3 id="prompt-engineering"&gt;Prompt engineering&lt;/h3&gt;
&lt;p&gt;Per addestrarlo mi sono servito di un prompt ridotto all’osso e di quattro brevi voci in testo semplice, scelte come esempi: fra queste, una di «medicina», una di «armi e armature» e una di «pittura».&lt;/p&gt;
&lt;h3 id="verifica"&gt;Verifica&lt;/h3&gt;
&lt;p&gt;Ho poi incollato un altro passo, estraneo alla sequenza iniziale:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Working neatly
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Never put down, if you can, two colors one on top of the other.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;But next, having made your design carefully, keep the place of shadows for them alone,
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&amp;amp; also separately that of lights &amp;amp; highlights, without layering one color all over &amp;amp; then highlighting or else shading on it.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;And in this way, you further your work, economize your colors &amp;amp; work neatly.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Which is the reason that, the colors not being muddled nor mixed together, they do not die &amp;amp; you soften the colors better, since they are not so thick.
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Il risultato è perfettamente in linea con il contenuto:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;categories&lt;/span&gt;&lt;span class="err"&gt;=&amp;#34;painting&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Se proviamo con una voce di una categoria che non figurava nemmeno fra i testi scelti per l’addestramento di GPT-3, la risposta sorprende.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;categories&lt;/span&gt;&lt;span class="err"&gt;=&amp;#34;jewelry&amp;#34;&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="risultato"&gt;Risultato&lt;/h3&gt;
&lt;p&gt;La categoria «jewelry» (gioielli) non esiste nella nostra edizione del Ms. Fr. 640: il gruppo di lavoro
quella, più ampia, di «Stones» (pietre). L’intuizione di GPT-3 è però buona, e lascia pensare che con un po’ di addestramento in più saprebbe categorizzare qualunque voce del Ms. Fr. 640, e forse anche quelle di testi tecnici cinquecenteschi affini.&lt;/p&gt;
&lt;h2 id="esperimento-2--marcatura-semantica"&gt;Esperimento 2 – Marcatura semantica&lt;/h2&gt;
&lt;p&gt;Alziamo l’asticella. Se un transformer come GPT-3 impara a categorizzare i testi secondo criteri editoriali specifici, saprà anche riconoscerne, almeno in parte, la marcatura?&lt;/p&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;&lt;em&gt;Secrets of Craft and Nature&lt;/em&gt; adotta una
di etichette semantiche e strutturali. Purtroppo GPT-3 non elabora immagini, a differenza di altri progetti come
. È probabile che le prossime versioni di GPT acquisiscano questa capacità, indispensabile per riconoscere la maggior parte degli aspetti strutturali e materiali di un documento. Lasceremo dunque da parte quei tag e ci concentreremo sulla marcatura che non richiede il riconoscimento delle immagini.&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 id="prompt-engineering-1"&gt;Prompt engineering&lt;/h3&gt;
&lt;p&gt;I tag semantici segnalano animali, piante, toponimi, percezioni sensoriali e altro ancora. Nel prompt di addestramento ho raccolto qualche esempio tratto dall’edizione:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Input prompt--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;The following is a list of words and their corresponding semantic tags
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cannons: &lt;span class="nt"&gt;&amp;lt;wp&amp;gt;&lt;/span&gt;cannons&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;flasks: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;flasks&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;wooden: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;wooden&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;iron: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;iron&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;parchment: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;parchment&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;goats: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;goats&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;lambs: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;lambs&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;leather: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;leather&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;earth: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;earth&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;fine fatty earth: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;fine fatty earth&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Venice: &lt;span class="nt"&gt;&amp;lt;pl&amp;gt;&lt;/span&gt;Venice&lt;span class="nt"&gt;&amp;lt;/pl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Flemish: &lt;span class="nt"&gt;&amp;lt;pl&amp;gt;&lt;/span&gt;Flemish&lt;span class="nt"&gt;&amp;lt;/pl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;almond: &lt;span class="nt"&gt;&amp;lt;pa&amp;gt;&lt;/span&gt;almond&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;almond oil: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;almond&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; oil&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;walnuts skin: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;walnuts&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; skin&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;molten lead: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;molten lead&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;today: &lt;span class="nt"&gt;&amp;lt;tmp&amp;gt;&lt;/span&gt;today&lt;span class="nt"&gt;&amp;lt;/tmp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;In the past: &lt;span class="nt"&gt;&amp;lt;tmp&amp;gt;&lt;/span&gt;In the past&lt;span class="nt"&gt;&amp;lt;/tmp&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Clockmakers: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;Clockmakers&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;red copper: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;red copper&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;crucible: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;crucible&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;bellows: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&lt;/span&gt;bellows&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;charcoal: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;charcoal&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;founders: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;founders&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="verifica-1"&gt;Verifica&lt;/h3&gt;
&lt;p&gt;Proviamo con il modello &lt;code&gt;Davinci-codex&lt;/code&gt; qualche parola facile: &lt;em&gt;Apothecary&lt;/em&gt;, &lt;em&gt;smoke&lt;/em&gt;, &lt;em&gt;glassmakers&lt;/em&gt;, &lt;em&gt;latten&lt;/em&gt; e &lt;em&gt;snake&lt;/em&gt;. Le risposte arrivano all’istante e sono impeccabili:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Apothecary: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;Apothecary&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;smoke: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;smoke&lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;glassmakers: &lt;span class="nt"&gt;&amp;lt;pro&amp;gt;&lt;/span&gt;glassmakers&lt;span class="nt"&gt;&amp;lt;/pro&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;latten: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;latten&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;snake: &lt;span class="nt"&gt;&amp;lt;al&amp;gt;&lt;/span&gt;snake&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Una prova più severa richiede parole composte – &lt;em&gt;copper plates&lt;/em&gt;, &lt;em&gt;walnut oil&lt;/em&gt;, &lt;em&gt;wood block&lt;/em&gt; – per vedere se GPT-3 se la cava con i tag annidati.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;copper plates: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;copper plates&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;walnut oil: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;pa&amp;gt;&lt;/span&gt;walnut&lt;span class="nt"&gt;&amp;lt;/pa&amp;gt;&lt;/span&gt; oil&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;wood block: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&lt;/span&gt;wood block&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Qui i risultati sono in chiaroscuro: &lt;code&gt;Davinci-codex&lt;/code&gt; ha etichettato correttamente solo &lt;em&gt;walnut oil&lt;/em&gt;, senza cogliere i tag &lt;code&gt;tl&lt;/code&gt; e &lt;code&gt;m&lt;/code&gt; annidati in &lt;em&gt;copper plates&lt;/em&gt; e &lt;em&gt;wood block&lt;/em&gt;. Ma, come mostra la prova successiva, basta un prompt di addestramento migliore per attenuare questi errori. Aggiunti altri cinque esempi di tag annidati, &lt;code&gt;Davinci-codex&lt;/code&gt; ha restituito un risultato quasi perfetto, con un solo errore (&lt;em&gt;oil paintbrushes&lt;/em&gt;):&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c"&gt;&amp;lt;!--Output--&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cannon powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;wp&amp;gt;&lt;/span&gt;cannon&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;arquebus powder: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;wp&amp;gt;&lt;/span&gt;arquebus&lt;span class="nt"&gt;&amp;lt;/wp&amp;gt;&lt;/span&gt; powder&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;oil paintbrushes: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;al&amp;gt;&lt;/span&gt;oil&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt; paintbrushes&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sheep footbones: &lt;span class="nt"&gt;&amp;lt;m&amp;gt;&amp;lt;al&amp;gt;&lt;/span&gt;sheep&lt;span class="nt"&gt;&amp;lt;/al&amp;gt;&lt;/span&gt; footbones&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;bronze mortar: &lt;span class="nt"&gt;&amp;lt;tl&amp;gt;&amp;lt;m&amp;gt;&lt;/span&gt;bronze&lt;span class="nt"&gt;&amp;lt;/m&amp;gt;&lt;/span&gt; mortar&lt;span class="nt"&gt;&amp;lt;/tl&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id="conclusione"&gt;Conclusione&lt;/h1&gt;
&lt;p&gt;Va ricordato che queste prove sono state condotte su frammenti di testo molto brevi. Sospetto che, dando più contesto negli esempi e nel prompt, i modelli GPT-3 farebbero ancora meglio; e un fine-tuning su insiemi di dati ad hoc migliorerebbe senza dubbio la precisione dell’etichettatura.&lt;br&gt;
Se per dimostrare l’affidabilità dei modelli linguistici pre-addestrati occorreranno ancora esperimenti su scala più ampia, si può nondimeno concludere che
&lt;mark&gt;questo approccio permette a chi cura un’edizione di automatizzare parecchie operazioni di annotazione in pochi passaggi, con un risparmio potenzialmente enorme di tempo e di denaro.&lt;/mark&gt;
&lt;/p&gt;</description></item><item><title>Visualizzare i manoscritti 2 (aggiornamento)</title><link>https://clementgodbarge.com/it/post/treemap2/</link><pubDate>Sat, 20 Nov 2021 16:00:00 +0000</pubDate><guid>https://clementgodbarge.com/it/post/treemap2/</guid><description>&lt;p&gt;Come promesso, ecco la nuova versione della treemap interattiva presentata in un
: questa volta con due modalità di visualizzazione.&lt;/p&gt;
&lt;div class="callout flex px-4 py-3 mb-6 rounded-md border-l-4 bg-blue-100 dark:bg-blue-900 border-blue-500"
data-callout="note"
data-callout-metadata=""&gt;
&lt;span class="callout-icon pr-3 pt-1 text-blue-600 dark:text-blue-300"&gt;
&lt;svg height="24" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"&gt;&lt;path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="m16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8l.8-2.685a4.5 4.5 0 0 1 1.13-1.897zm0 0L19.5 7.125"/&gt;&lt;/svg&gt;
&lt;/span&gt;
&lt;div class="callout-content dark:text-neutral-300"&gt;
&lt;div class="callout-title font-semibold mb-1"&gt;Nota&lt;/div&gt;
&lt;div class="callout-body"&gt;&lt;p&gt;Per una resa migliore, impostate la pagina in modalità chiara (cliccate sull’icona della luna in alto a destra).&lt;/p&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;head&gt;
&lt;meta charset="UTF-8" /&gt;
&lt;meta http-equiv="X-UA-Compatible" content="IE=edge" /&gt;
&lt;meta name="viewport" content="width=device-width, initial-scale=1.0" /&gt;
&lt;title&gt;&lt;/title&gt;
&lt;link rel="preconnect" href="https://fonts.gstatic.com" /&gt;
&lt;link
href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;700&amp;display=swap"
rel="stylesheet"
/&gt;
&lt;link rel="stylesheet" href="css/index.css" /&gt;
&lt;link rel="stylesheet" href="css/vis-treemap.css" /&gt;
&lt;link rel="stylesheet" href="css/vis-tooltip.css" /&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;div class="stacked"&gt;
&lt;div class="switch"&gt;
&lt;input
type="checkbox"
name="group-by-category-switch"
id="group-by-category-switch"
checked
/&gt;
&lt;label for="group-by-category-switch"&gt; Group folios by category &lt;/label&gt;
&lt;/div&gt;
&lt;div id="treemap"&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;script src="https://cdn.jsdelivr.net/npm/d3@7.9.0/dist/d3.min.js" integrity="sha384-CjloA8y00+1SDAUkjs099PVfnY2KmDC2BZnws9kh8D/lX1s46w6EPhpXdqMfjK6i" crossorigin="anonymous" referrerpolicy="no-referrer"&gt;&lt;/script&gt;
&lt;script src="js/vis-treemap.js"&gt;&lt;/script&gt;
&lt;script src="js/vis-tooltip.js"&gt;&lt;/script&gt;
&lt;script src="js/index.js"&gt;&lt;/script&gt;
&lt;/body&gt;</description></item><item><title>L’archivio a colpo d’occhio</title><link>https://clementgodbarge.com/it/post/dashboard/</link><pubDate>Mon, 24 May 2021 16:00:00 +0000</pubDate><guid>https://clementgodbarge.com/it/post/dashboard/</guid><description>&lt;h1 id="il-problema"&gt;Il problema&lt;/h1&gt;
&lt;p&gt;Gli archivi storici possono essere di un disordine che scoraggia. Il &lt;em&gt;Mediceo del Principato&lt;/em&gt; dell’
ne è l’esempio perfetto: solo una piccola parte del fondo è inventariata, e molti documenti sono sparsi senza ragione apparente in più di 6.500 volumi. Come se non bastasse, l’archivio consente di consultare soltanto un numero limitato di volumi – o &lt;em&gt;filze&lt;/em&gt;, come si dice a Firenze. In tempi normali il limite è di 4 &lt;em&gt;filze&lt;/em&gt; al giorno; in tempo di pandemia è sceso a 4 ogni due settimane. In mancanza di inventari dettagliati, la mole dell’archivio obbliga chi vi fa ricerca a escogitare strategie per arrivare in fretta ai documenti che cerca.&lt;/p&gt;
&lt;h1 id="la-soluzione"&gt;La soluzione&lt;/h1&gt;
&lt;p&gt;C’è chi si affida al caso, e chi prova a fare congetture ragionate su cronologia, destinatari, autori, provenienza del fondo, lingua e via dicendo. Ma &lt;em&gt;guardare&lt;/em&gt; tutte queste variabili insieme può far emergere regolarità inattese nella struttura dell’archivio e affinare le congetture. La mia esperienza dice che i metadati che i ricercatori raccolgono di solito in un foglio di calcolo, una volta messi in grafico, aiutano sensibilmente a orientarsi in archivio: si sa dove si è, e che cosa si sta cercando.&lt;/p&gt;
&lt;h1 id="lesperimento"&gt;L’esperimento&lt;/h1&gt;
&lt;p&gt;La mia ricerca attuale verte sulla corrispondenza di una spia del Cinquecento. Le sue lettere sono disperse in centinaia di &lt;em&gt;filze&lt;/em&gt;: scritte sotto identità diverse, a destinatari diversi e talvolta inaspettati, da luoghi diversi, e così via. Per individuare le &lt;em&gt;filze&lt;/em&gt; in cui è più probabile trovarle, ho messo in piedi una dashboard, cioè un’applicazione web di visualizzazione interattiva dei dati (
) che collega informazioni di ogni genere – geografiche, cronologiche – a un diagramma gerarchico (
) del fondo archivistico. A colpo d’occhio la dashboard mi dice che cosa è già stato trovato, quanto pesa sul totale, e mi dà un’idea approssimativa di dove potrei cercare lettere nuove. Cliccando su una variabile, poi, tutti i diagrammi si aggiornano e mostrano le correlazioni corrispondenti.&lt;/p&gt;
&lt;h1 id="prossimi-passi"&gt;Prossimi passi&lt;/h1&gt;
&lt;p&gt;Ma forse la cosa più importante è che la dashboard può trasformarsi in un indice visivo. Quando l’edizione critica delle lettere sarà online, farà da porta d’ingresso alternativa, da cui i lettori potranno esplorare i dati. Per ragioni di riservatezza al momento posso mostrare soltanto una schermata parzialmente oscurata, ma l’anno prossimo pubblicherò la dashboard completa; nel frattempo sarà presto disponibile un prototipo. A presto!&lt;/p&gt;</description></item><item><title>Visualizzare la marcatura semantica del BnF Ms. Fr. 640</title><link>https://clementgodbarge.com/it/post/visualization/</link><pubDate>Sun, 20 Dec 2020 18:15:00 +0000</pubDate><guid>https://clementgodbarge.com/it/post/visualization/</guid><description>&lt;h1 id="panoramica"&gt;Panoramica&lt;/h1&gt;
&lt;p&gt;Le edizioni critiche ricche di dati contengono annotazioni editoriali preziose, che si possono estrarre, analizzare e visualizzare per gli scopi più diversi. È il caso di
, uscita nel 2020, che mette a disposizione il proprio file di metadati sul suo repository GitHub. In questo post mostro come raccogliere tutte quelle variabili in una matrice di correlazione e visualizzarle in più modi.&lt;/p&gt;
&lt;h1 id="i-dati"&gt;I dati&lt;/h1&gt;
&lt;p&gt;Il Making and Knowing Project produce un foglio di calcolo con informazioni aggiornate sul contenuto del manoscritto: &lt;code&gt;entry_metadata.csv&lt;/code&gt;. Il file si scarica dal
del Making &amp;amp; Knowing; in alternativa, si possono generare file .csv su misura, con più marcatura, grazie all’eccellente
di Matthew Kumar, una versione Python del BnF Ms. Fr. 640.&lt;/p&gt;
&lt;h2 id="preparare-python"&gt;Preparare Python&lt;/h2&gt;
&lt;p&gt;Useremo Pandas per la manipolazione dei dati, Matplotlib e seaborn per le heatmap, e infine NetworkX per costruire reti basate sulle correlazioni.&lt;br&gt;
Con variabili di questo tipo eviteremo il metodo di Pearson e useremo invece il metodo 𝜙𝐾. Vi conviene
su questo metodo di correlazione e su &lt;code&gt;PhiK&lt;/code&gt;, la libreria corrispondente.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;#install packages&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="n"&gt;phik&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# import modules&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;pandas&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;pd&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;plt&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;seaborn&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;sns&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;networkx&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;nx&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id="preparare-i-dati"&gt;Preparare i dati&lt;/h2&gt;
&lt;p&gt;Per prima cosa scarichiamo dal
dell’edizione, nella cartella metadata, l’ultimo file di metadati.
Terremo solo le colonne che ci servono. Per questa dimostrazione prendo tutti i tag semantici della traduzione inglese &lt;code&gt;tl&lt;/code&gt;, ma potete scegliere anche quelli della trascrizione francese &lt;code&gt;tc&lt;/code&gt; o della versione normalizzata &lt;code&gt;tcn&lt;/code&gt;.
I dati arrivano come valori separati da punto e virgola, e dobbiamo farli contare a Python: useremo il metodo stack-unstack con l’espressione regolare &lt;code&gt;[^;\s][^\;]*[^;\s]*&lt;/code&gt;.
Per rendere la matrice più leggibile rinominiamo ogni colonna. Se avete fretta potete saltare questo passaggio; tenete solo a mente che a questo punto il nostro dataframe si chiama &lt;code&gt;tagsrn&lt;/code&gt;.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# load the edition&amp;#39;s metadata&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;df&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;read_csv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;entry_metadata.csv&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# select the tags you want to correlate&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;dftags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;df&lt;/span&gt;&lt;span class="p"&gt;[[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;al_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;bp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;cn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;df_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;env_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;m_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;md_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;ms_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;mu_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pa_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pro_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tmp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;wp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;de_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;el_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;it_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;la_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;oc_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;po_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# count comma separated values&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;tagcount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;dftags&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dropna&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;str&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;[^;\s][^\;]*[^;\s]*&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;unstack&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# rename columns&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;tagsrn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tagcount&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rename&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;columns&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;al_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;animals&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;bp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;body parts&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;cn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;currency&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;df_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;definitions&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;env_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;environment&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;m_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;material&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;md_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;medical&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;ms_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;measurement&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;mu_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;music&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pa_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;plant&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;toponym&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;person&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;pro_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;profession&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sn_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;sensory&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tl_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tool&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;tmp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;temporal&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;wp_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;weapons&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;de_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;German&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;el_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Greek&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;it_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Italian&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;la_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Italian&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;oc_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Occitan&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;po_tl&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Poitevin&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id="correlare"&gt;Correlare&lt;/h1&gt;
&lt;p&gt;Pulito il dataframe, si passa a calcolare i coefficienti di correlazione fra le variabili. A questo punto è essenziale conoscere i propri dati e scegliere il metodo di correlazione più adatto; il pacchetto &lt;code&gt;pandas-profiling&lt;/code&gt; è di grande aiuto.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# calculate correlation coefficient with the phi k method&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;cortag&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tagsrn&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;phik_matrix&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;cortag&lt;/code&gt; è la nostra matrice di correlazione. Possiamo ora provare vari tipi di visualizzazione.&lt;/p&gt;
&lt;h1 id="visualizzare"&gt;Visualizzare&lt;/h1&gt;
&lt;p&gt;La prima cosa da provare è visualizzarla come matrice a colori, con il
di &lt;code&gt;seaborn&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id="heatmap-di-correlazione"&gt;Heatmap di correlazione&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ax&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;subplots&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;figsize&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;ax&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sns&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;heatmap&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;linewidths&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;.03&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vmin&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cmap&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Oranges&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;square&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Heatmap di correlazione del BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/heatmap_hu_943abdd98e24b0a3.webp 320w, https://clementgodbarge.com/post/visualization/heatmap_hu_eca41311a0e0ac17.webp 480w, https://clementgodbarge.com/post/visualization/heatmap_hu_1650defb643c0285.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/heatmap_hu_943abdd98e24b0a3.webp"
width="760"
height="665"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Se conoscete bene il testo, vedrete subito che la heatmap torna. I nomi di persona, per esempio, sono fortemente correlati al latino: era consuetudine, soprattutto fra gli umanisti del Cinquecento, latinizzarli.&lt;/p&gt;
&lt;p&gt;Qualcuno obietterà che la heatmap non fa che dire l’ovvio. Non ha tutti i torti; e a prima vista i tag medici sembrano dargli ragione, dato che si correlano, com’era prevedibile, con parti del corpo, misure e piante.&lt;/p&gt;
&lt;p&gt;Ma a leggere la heatmap con più attenzione, riga per riga, saltano fuori correlazioni interessanti e inattese. Che i tag medici siano correlati con le parole italiane e latine, per esempio, dice qualcosa sull’origine delle ricette mediche del Ms. Fr. 640. Allo stesso modo, la correlazione fra professioni, definizioni e misure mostra quanto l’identità professionale strutturi il discorso tecnico del Cinquecento.&lt;/p&gt;
&lt;h3 id="clustermap-di-correlazione"&gt;Clustermap di correlazione&lt;/h3&gt;
&lt;p&gt;Le heatmap servono in fase «esplorativa», ma al vostro pubblico possono sembrare confuse, soprattutto se state discutendo – o ancora cercando – raggruppamenti semantici precisi nel manoscritto. Il modulo &lt;code&gt;clustermap&lt;/code&gt; di seaborn può dare risultati interessanti.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;clustermap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sns&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clustermap&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;figsize&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;13&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;dendrogram_ratio&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;.1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.2&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;vmin&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cmap&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Oranges&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cbar_pos&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;.06&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.03&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;.68&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Clustermap di correlazione del BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/clustermap_hu_677e6e5028f9d902.webp 320w, https://clementgodbarge.com/post/visualization/clustermap_hu_2dac945fa40a8ac6.webp 480w, https://clementgodbarge.com/post/visualization/clustermap_hu_3abaf7b36a53572.webp 753w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/clustermap_hu_677e6e5028f9d902.webp"
width="753"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Oltre a somigliare a un insetto pixelato (sì, la parola è nell’OED), la clustermap separa nettamente i tag isolati (in alto e a sinistra) da quelli più interconnessi. Si distinguono anche raggruppamenti isolati, come musica e poitevino (chi l’avrebbe detto!), da altri più centrali come misure, materiali, definizioni e armi. Le professioni sono più interconnesse, ma non appartengono, almeno in questa matrice, a un raggruppamento particolare.&lt;/p&gt;
&lt;h3 id="rete-di-correlazione"&gt;Rete di correlazione&lt;/h3&gt;
&lt;p&gt;Se vogliamo condensare ancora di più le correlazioni della matrice, i grafi offrono una soluzione elegante, soprattutto quando si tratta di comunicare il contenuto del manoscritto.&lt;br&gt;
Occorre trasformare la matrice in una lista di archi e nodi, e fissare una soglia per escludere dal grafo le correlazioni più deboli.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# transform the data&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cortag&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stack&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reset_index&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;columns&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# threshold &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;loc&lt;/span&gt;&lt;span class="p"&gt;[(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;.6&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;])]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# create edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;G&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;from_pandas_edgelist&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links_filtered&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# draw network using Kamada &amp;amp; Kawai&amp;#39;s algorithm &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;figure&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="n"&gt;figsize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;draw_kamada_kawai&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;G&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;with_labels&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node_color&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;red&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;edge_color&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;black&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;linewidths&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;font_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Grafo di correlazione del BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/graph_hu_396032c00cb7f7f3.webp 320w, https://clementgodbarge.com/post/visualization/graph_hu_85295e5fba6293bc.webp 480w, https://clementgodbarge.com/post/visualization/graph_hu_4190dbca7e86ffc1.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/graph_hu_396032c00cb7f7f3.webp"
width="760"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;Se archi e nodi sono troppi, si può sempre ritoccare la soglia per un risultato più pulito. Altrimenti si esporta il grafo per lavorarci in &lt;code&gt;Gephi&lt;/code&gt;, con la funzione &lt;code&gt;.write_gexf()&lt;/code&gt;.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;write_gexf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;G&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;graph.gexf&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Il risultato lo vedete in apertura di questo post.&lt;/p&gt;
&lt;h3 id="aggiornamento-rete-circolare-pesata"&gt;Aggiornamento: rete circolare pesata&lt;/h3&gt;
&lt;p&gt;Cercavo un modo di rappresentare le matrici di correlazione come reti pesate, e ho trovato questo approccio interessante
, che adatto qui al nostro insieme di dati.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# create graph weighted by correlation coefficients (unfiltered)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Gx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;nx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;from_pandas_edgelist&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;links&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var1&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;var2&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;edge_attr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# determine a threshold to remove some edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;threshold&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;0.4&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# list to store edges to remove&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;remove&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# loop through edges in Gx and find correlations which are below the threshold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;var2&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;edges&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;corr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;var2&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;value&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="c1"&gt;#add to remove node list if abs(corr) &amp;lt; threshold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;corr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;threshold&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;append&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;var1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;var2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# remove edges contained in the remove list&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;remove_edges_from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39; edges removed&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Tolti alcuni archi, possiamo stabilirne colore e spessore.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# determine the colors of edges&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_colour&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;correlation&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mf"&gt;0.8&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;#ff872c&amp;#39;&lt;/span&gt; &lt;span class="c1"&gt;# orange&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;#f11d28&amp;#39;&lt;/span&gt; &lt;span class="c1"&gt;# red&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_thickness&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;benchmark_thickness&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;benchmark_thickness&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;correlation&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;scaling_factor&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;assign_node_size&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;degree&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;degree&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scaling_factor&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Diamo inoltre ai nodi una dimensione proporzionale al numero di connessioni.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# assign node size depending on number of connections (degree)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;node_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Gx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;degree&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;items&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;node_size&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;assign_node_size&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Il risultato è un grafo pesato che ammette più nodi e molti più archi, restando leggibile e informativo.&lt;/p&gt;
&lt;p&gt;
&lt;figure &gt;
&lt;div class="flex justify-center "&gt;
&lt;div class="w-full" &gt;
&lt;img alt="Grafo di correlazione pesato del BnF Ms. Fr. 640"
srcset="https://clementgodbarge.com/post/visualization/weightedgraph_hu_7b75f2e5fc7edb1b.webp 320w, https://clementgodbarge.com/post/visualization/weightedgraph_hu_9f0550eb49c17f11.webp 480w, https://clementgodbarge.com/post/visualization/weightedgraph_hu_a9106989817361fb.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://clementgodbarge.com/post/visualization/weightedgraph_hu_7b75f2e5fc7edb1b.webp"
width="760"
height="760"
loading="lazy" data-zoomable /&gt;&lt;/div&gt;
&lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;</description></item><item><title>La corrispondenza segreta di Filippo Cavriana, 1568—1589.</title><link>https://clementgodbarge.com/it/project/cavrianas-correspondence/</link><pubDate>Fri, 27 Dec 2019 00:00:00 +0000</pubDate><guid>https://clementgodbarge.com/it/project/cavrianas-correspondence/</guid><description>&lt;p&gt;Le lettere segrete di Filippo Cavriana sono una testimonianza senza pari sulle guerre di religione francesi: per più di vent’anni seguono gli intrighi, le battaglie, i negoziati e le congiure che scandivano la vita alla corte di Francia, e non solo.&lt;/p&gt;
&lt;p&gt;Conservate all’
, le lettere sono in gran parte disperse in centinaia di fasci non inventariati – le &lt;em&gt;filze&lt;/em&gt;, come si chiamano a Firenze. Negli ultimi anni ne ho scoperte decine di nuove, ma questo genere di ricerca d’archivio si fa sempre più arduo: la distanza da Firenze, il degrado di alcuni servizi pubblici in Italia e la pandemia di Covid-19 concorrono a fare di un progetto di pubblicazione come questo un’impresa lunga e incerta.&lt;/p&gt;
&lt;p&gt;Pubblicare fonti primarie dovrebbe restare, nonostante tutto, parte del mestiere dello storico. La forma di queste pubblicazioni, però, dovrebbe probabilmente cambiare: non solo adattarsi alle condizioni che ho appena descritto, ma rispecchiare meglio la natura incrementale di questa ricerca, e permettere a qualcun altro di continuarla.&lt;/p&gt;
&lt;p&gt;Da qui l’idea di un nuovo tipo di pubblicazione digitale: &lt;strong&gt;scalabile&lt;/strong&gt;, &lt;strong&gt;sostenibile&lt;/strong&gt;, &lt;strong&gt;affidabile&lt;/strong&gt;, &lt;strong&gt;indipendente dalla piattaforma&lt;/strong&gt; e, potenzialmente, &lt;strong&gt;collaborativa&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;L’edizione è ora in rete: &lt;strong&gt;
&lt;/strong&gt;. Raccoglie le lettere trascritte finora, dagli Archivi di Stato di Firenze e di Mantova e dalla BnF, e cresce man mano che ne affiorano di nuove. Le ragioni della sua architettura le ho spiegate in
.&lt;/p&gt;</description></item></channel></rss>