{"id":299,"date":"2018-02-15T18:44:20","date_gmt":"2018-02-15T17:44:20","guid":{"rendered":"http:\/\/www.julienrossi.com\/blog\/?p=299"},"modified":"2018-02-15T18:44:20","modified_gmt":"2018-02-15T17:44:20","slug":"travailler-avec-des-pdf-sous-python","status":"publish","type":"post","link":"https:\/\/www.julienrossi.com\/blog\/2018\/02\/15\/travailler-avec-des-pdf-sous-python\/","title":{"rendered":"Travailler avec des PDF sous Python"},"content":{"rendered":"<p><em>Ce billet de blog s&#8217;inscrit dans la s\u00e9rie \u00ab Voyage au pays de la grosse donn\u00e9e \u00bb<\/em><em> qui cherche \u00e0 voir ce qu&#8217;il est possible de faire en sciences sociales avec du traitement automatique de texte et du langage sur des volumes de donn\u00e9es massives et parfois h\u00e9t\u00e9rog\u00e8nes. Il n&#8217;a pas la pr\u00e9tention de montrer l&#8217;\u00e9tat de l&#8217;art dans le domaine, mais cherche \u00e0 parler des difficult\u00e9s que l&#8217;on peut rencontrer en s&#8217;embarquant dans ce type de recherche, pour voir quelles peuvent \u00eatre les opportunit\u00e9s et les limites de telles m\u00e9thodes pour le chercheur moyen que je suis, aux comp\u00e9tences en informatique limit\u00e9es.<\/em><\/p>\n<p>Qui a fait du lobbying sur l&#8217;article 89 du R\u00e8glement g\u00e9n\u00e9ral de protection des donn\u00e9es, qui d\u00e9finit le r\u00e9gime d\u00e9rogatoire applicable aux traitements de donn\u00e9es \u00e0 caract\u00e8re personnel pour des finalit\u00e9s de recherche <span class=\"footnote_referrer\" ><a><sup id=\"footnote_plugin_tooltip_299_1\" class=\"footnote_plugin_tooltip_text\" onclick=\"footnote_moveToAnchor_299('footnote_plugin_reference_299_1');\" >[1]<\/sup ><\/a><span id=\"footnote_plugin_tooltip_text_299_1\" class=\"footnote_tooltip\" >Une voire une cat\u00e9gorie d&#8217;articles sur ce sujet devrait \u00eatre d\u00e9di\u00e9 bient\u00f4t sur mon blog. En attendant, vous pouvez aller voir le compte-rendu d&#8217;une journ\u00e9e d&#8217;\u00e9tude organis\u00e9e&nbsp;&#x2026; <span class=\"footnote_tooltip_continue\" onclick=\"footnote_moveToAnchor_299('footnote_plugin_reference_299_1');\">Continue reading<\/span><\/span ><\/span><script type=\"text\/javascript\"> jQuery('#footnote_plugin_tooltip_299_1').tooltip({ tip: '#footnote_plugin_tooltip_text_299_1', tipClass: 'footnote_tooltip', effect: 'fade', predelay: 0, fadeInSpeed: 200, delay: 400, fadeOutSpeed: 200, position: 'top center', relative: true, offset: [-7, 0], });<\/script>.<\/p>\n<p>D&#8217;o\u00f9 tire-t-il sa l\u00e9gitimit\u00e9 ? Quels discours, quelles strat\u00e9gies argumentatives ont \u00e9t\u00e9 adopt\u00e9es par quels acteurs pour influer sur son contenu ?<\/p>\n<p>Que ce soit <a href=\"https:\/\/ec.europa.eu\/home-affairs\/what-is-new\/public-consultation\/2009\/consulting_0003_en\">sur le site de la Commission europ\u00e9enne<\/a> (documents envoy\u00e9s pour une consultation publique datant de 2009), sur <a href=\"http:\/\/www.europarl.europa.eu\/oeil\/popups\/ficheprocedure.do?reference=2012\/0011(COD)&amp;l=fr\">celui du Parlement europ\u00e9en<\/a> (rapports parlementaires, amendements d\u00e9pos\u00e9s, etc.) ou bien gr\u00e2ce au travaille des militants et militantes ayant cr\u00e9\u00e9 <a href=\"https:\/\/www.lobbyplag.eu\/\">le site Internet Lobbyplag<\/a>, de nombreux documents relatifs \u00e0 ce sujet sont disponibles.<\/p>\n<p>Ces documents, sont, dans leur ensemble, au format PDF.<\/p>\n<p>Pour l&#8217;instant, j&#8217;ai ainsi r\u00e9cup\u00e9r\u00e9 pr\u00e8s de 300 documents de diff\u00e9rentes institutions publiques, lobbies, ONG, et particuliers exprimant leurs positions sur la r\u00e9forme du droit des donn\u00e9es \u00e0 caract\u00e8re personnel :<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-20-35.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-300\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-20-35.png\" alt=\"\" width=\"485\" height=\"262\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-20-35.png 485w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-20-35-300x162.png 300w\" sizes=\"auto, (max-width: 485px) 100vw, 485px\" \/><\/a><\/p>\n<p>Gr\u00e2ce \u00e0 la librairie PyPDF2 il est possible de faire lire par l&#8217;ordinateur tous ces documents PDF \u00e0 la recherche de certains mots-clefs (ici : science, scientific, research, data mining et big data) pour voir dans quels documents ces mots sont utilis\u00e9s et m\u00eame pour indiquer \u00e0 quelle page on peut les trouver.<\/p>\n<p style=\"padding-left: 30px;\"><span style=\"color: #008000;\"><strong>[Utile]<\/strong><\/span> Cela ne permet pas de faire de l&#8217;analyse automatique, mais permet de gagner beaucoup de temps dans le tri d&#8217;un corpus pertinent \u00e0 une analyse qualitative classique<\/p>\n<p>&nbsp;<\/p>\n<p>Voici le tableau ainsi g\u00e9n\u00e9r\u00e9 :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-35-35.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-304\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-35-35.png\" alt=\"\" width=\"581\" height=\"149\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-35-35.png 581w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-35-35-300x77.png 300w\" sizes=\"auto, (max-width: 581px) 100vw, 581px\" \/><\/a>Ainsi qu&#8217;on peut le constater, certains documents ne contiennent pas le mot \u00ab scientific \u00bb et il serait alors possible d&#8217;exclure de mon corpus ces documents.<\/p>\n<p>Mais bizarrement, certains documents PDF qui contiennent bien, quand je l&#8217;ouvre, le mot &#8220;scientific&#8221;, ne sont pas d\u00e9tect\u00e9s par mon script. C&#8217;est ainsi le cas de <a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/LIBE.pdf\">LIBE.pdf<\/a>, le rapport de la commission Libert\u00e9s civiles, justice et affaires int\u00e9rieures :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-39-43.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-306 size-thumbnail aligncenter\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-39-43-150x150.png\" alt=\"\" width=\"150\" height=\"150\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-41-38.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-307 alignright\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-41-38.png\" alt=\"\" width=\"1004\" height=\"45\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-41-38.png 1004w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-41-38-300x13.png 300w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-41-38-768x34.png 768w\" sizes=\"auto, (max-width: 1004px) 100vw, 1004px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>Pourtant, aucun message d&#8217;erreur n&#8217;avait \u00e9t\u00e9 g\u00e9n\u00e9r\u00e9 : le document semble avoir \u00e9t\u00e9 ouvert correctement par le script. O\u00f9 se situe l&#8217;erreur de lecture ? D&#8217;autres documents sont-ils affect\u00e9s ? Une v\u00e9rification d\u00e9taill\u00e9e s&#8217;impose alors, qui permet de montrer qu&#8217;en effet, PyPDF2, la librairie que j&#8217;avais utilis\u00e9e, se r\u00e9v\u00e8le totalement incapable de lire le document PDF en question. L\u00e0 o\u00f9 normalement je devrais obtenir ce type de r\u00e9sultat :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-01.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-308\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-01.png\" alt=\"\" width=\"628\" height=\"202\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-01.png 628w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-01-300x96.png 300w\" sizes=\"auto, (max-width: 628px) 100vw, 628px\" \/><\/a><\/p>\n<p>J&#8217;obtiens ceci, et ce alors m\u00eame que le petit message &#8220;False&#8221; m&#8217;indique que mon PDF n&#8217;est nullement chiffr\u00e9 :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-49.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-309\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-49.png\" alt=\"\" width=\"620\" height=\"81\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-49.png 620w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-14-44-49-300x39.png 300w\" sizes=\"auto, (max-width: 620px) 100vw, 620px\" \/><\/a><\/p>\n<p>En poussant un peu mes tests, je me suis rendu compte que 72 documents sur 266 \u00e9tait affect\u00e9, soit presque un tiers (27%) de mon corpus. Un probl\u00e8me majeur de nature \u00e0 biaiser totalement le corpus final que j&#8217;aurai \u00e0 analyser.<\/p>\n<p style=\"padding-left: 30px;\"><span style=\"color: #800000;\"><strong>[Probl\u00e8me]<\/strong><\/span> Certaines erreurs sont d\u00e9tect\u00e9es par l&#8217;ordinateur. Par exemple, il peut arriver qu&#8217;un document ait pour extension &#8220;.pdf&#8221; et qu&#8217;il soit incapable de l&#8217;ouvrir car le fichier est corrompu ou bien est par exemple un fichier .doc dont l&#8217;extension a \u00e9t\u00e9 chang\u00e9e par erreur. Cependant, sauf si on lui demande de v\u00e9rifier en lui fournissant un dictionnaire ou une expression dont on estime qu&#8217;elle doit se trouver absolument dans le texte du PDF (j&#8217;ai utilis\u00e9 dans mon corpus le terme &#8220;data&#8221; pour ce test) l&#8217;ordinateur est incapable de savoir si une cha\u00eene de texte extraite d&#8217;un PDF signifie quelque chose pour un humain ou pas. Et il n&#8217;affichera aucun message d&#8217;erreur pour alerter l&#8217;utilisateur du programme qu&#8217;il n&#8217;a pas pu lire certains fichiers. Sans une v\u00e9rification manuelle par le chercheur, des biais de s\u00e9lection pouvant invalider compl\u00e8tement la fiabilit\u00e9 des r\u00e9sultats peuvent facilement se produire.<\/p>\n<p>&nbsp;<\/p>\n<p>Que faire face \u00e0 cela ? Le <a href=\"https:\/\/code.tutsplus.com\/tutorials\/how-to-work-with-pdf-documents-using-python--cms-25726\">seul site web<\/a> que j&#8217;ai trouv\u00e9 qui parle du probl\u00e8me ne propose aucune solution avec la librairie PyPDF2. Comment se comporte une autre librairie ?<\/p>\n<p>Il existe heureusement une autre librairie, PDFMiner. Et <a href=\"https:\/\/stackoverflow.com\/questions\/26494211\/extracting-text-from-a-pdf-file-using-pdfminer-in-python\">gr\u00e2ce \u00e0 DuckPuncher de Stackoverflow.com<\/a>, j&#8217;ai m\u00eame pu r\u00e9cup\u00e9rer une fonction pr\u00eate \u00e0 l&#8217;emploi pour effectuer l&#8217;extraction de texte du PDF. Et le miracle se produisit !<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-15-05-55.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-311\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-15-05-55.png\" alt=\"\" width=\"577\" height=\"205\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-15-05-55.png 577w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-15-05-55-300x107.png 300w\" sizes=\"auto, (max-width: 577px) 100vw, 577px\" \/><\/a><\/p>\n<p>Gr\u00e2ce \u00e0 cette autre librairie, je n&#8217;ai plus que 39 documents sur 266, soit environ 15%, qui ne contiennent m\u00eame pas, selon mon script, une seule fois le mot \u00ab data \u00bb.<\/p>\n<p>Certains documents PDF, comme celui-ci, r\u00e9sistent donc h\u00e9las m\u00eame \u00e0 PDFMiner :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-50-27.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-316\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-50-27.png\" alt=\"\" width=\"602\" height=\"182\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-50-27.png 602w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-50-27-300x91.png 300w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/a><\/p>\n<p>Pour ces documents, aucun espoir, m\u00eame le lecteur de PDF XReader est incapable de faire une recherche dans leur texte :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-45-41-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-314\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-45-41-1-300x101.png\" alt=\"\" width=\"450\" height=\"151\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-45-41-1-300x101.png 300w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-45-41-1-768x258.png 768w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/Capture-du-2018-02-15-16-45-41-1.png 903w\" sizes=\"auto, (max-width: 450px) 100vw, 450px\" \/><\/a><\/p>\n<p>Mais sur ces 39 documents qui ne contiennent pas le mot data, certains sont en r\u00e9alit\u00e9 r\u00e9dig\u00e9s dans une autre langue que l&#8217;anglais, par exemple en allemand. En enlevant ces 22 fichiers, avec un tri \u00e0 la main (m\u00eame s&#8217;il aurait pu \u00eatre possible avec scipy de faire une d\u00e9tection automatique de langage) il reste 17 fichiers que PDFMiner ne sait pas lire dont 6 qui ne contiennent que des images, et 1 document qui ne contient pas le mot &#8220;data&#8221;. Donc sur 244 documents en anglais, le taux d&#8217;\u00e9chec tombe \u00e0 7%. \u00c7a reste beaucoup, mais c&#8217;est mieux que les 30% de tout \u00e0 l&#8217;heure !<\/p>\n<p>Il reste donc 17 fichiers \u00e0 fouiller manuellement pour savoir s&#8217;ils contiennent un des mots-clefs recherch\u00e9s et savoir s&#8217;il faut ou non les int\u00e9grer \u00e0 notre corpus.<\/p>\n<p>Quant au 227 documents en anglais lus par PDFMiner, voici combien d&#8217;entre eux contiennent les mots recherch\u00e9s :<\/p>\n<p><a href=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/graphique.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-319\" src=\"http:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/graphique.png\" alt=\"\" width=\"605\" height=\"340\" srcset=\"https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/graphique.png 605w, https:\/\/www.julienrossi.com\/blog\/wp-content\/uploads\/2018\/02\/graphique-300x169.png 300w\" sizes=\"auto, (max-width: 605px) 100vw, 605px\" \/><\/a>Le terme \u00ab research \u00bb a donc souvent \u00e9t\u00e9 utilis\u00e9 sans \u00eatre corr\u00e9l\u00e9 \u00e0 \u00ab science \u00bb ou \u00ab scientific \u00bb. Dans quel autre contexte a-t-il \u00e9t\u00e9 utilis\u00e9 ?<\/p>\n<p>De fa\u00e7on \u00e9tonnante, le terme \u00ab big data \u00bb n&#8217;a \u00e9t\u00e9 utilis\u00e9 que 2 fois, et \u00ab data mining \u00bb 5 fois. Est-ce \u00e0 dire que les repr\u00e9sentations politiques li\u00e9es au Big Data aient \u00e9t\u00e9 exclues du d\u00e9bat ? Lorsque l&#8217;on conna\u00eet de fa\u00e7on empirique ce terrain, on sait qu&#8217;une telle conclusion serait \u00e9videmment fausse.<\/p>\n<p>Par contre c&#8217;est int\u00e9ressant de noter que seuls une cinquantaine de documents contiennent le mot \u00ab scientific \u00bb, sur 244, soit 21,5%.<\/p>\n<p>Ce qui est pas mal, mais permet d\u00e9j\u00e0 de circonscrire un corpus plus restreint sur lequel concentrer une analyse qualitative de contenu.<\/p>\n<p><span style=\"text-decoration: underline;\">Pour information, le script utilis\u00e9 \u00e0 l&#8217;origine \u00e9tait le suivant<\/span> :<\/p>\n<p>&nbsp;<\/p>\n<div class=\"codecolorer-container python default\" style=\"overflow:auto;white-space:nowrap;width:435px;height:300px;\"><div class=\"python codecolorer\"><span class=\"co1\">#!\/usr\/bin\/env python<\/span><br \/>\n<span class=\"co1\"># -*- coding: utf-8 -*-<\/span><br \/>\n<br \/>\n<span class=\"kw1\">import<\/span> unicodecsv <span class=\"kw1\">as<\/span> <span class=\"kw3\">csv<\/span><br \/>\n<span class=\"kw1\">import<\/span> pyPdf<span class=\"sy0\">,<\/span> <span class=\"kw3\">os<\/span><span class=\"sy0\">,<\/span> <span class=\"kw3\">sys<\/span><br \/>\n<br \/>\n<br \/>\n<br \/>\n<span class=\"kw1\">if<\/span> __name__ <span class=\"sy0\">==<\/span> <span class=\"st0\">&quot;__main__&quot;<\/span>:<br \/>\n&nbsp; &nbsp; list_of_dirs <span class=\"sy0\">=<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">listdir<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; headers <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;folder&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; wordlist <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; lines <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; f <span class=\"sy0\">=<\/span> <span class=\"kw2\">open<\/span><span class=\"br0\">&#40;<\/span><span class=\"st0\">&quot;output.csv&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;w&quot;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; writer <span class=\"sy0\">=<\/span> <span class=\"kw3\">csv<\/span>.<span class=\"me1\">DictWriter<\/span><span class=\"br0\">&#40;<\/span>f<span class=\"sy0\">,<\/span> fieldnames<span class=\"sy0\">=<\/span>headers<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; writer.<span class=\"me1\">writeheader<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; <span class=\"kw1\">for<\/span> element <span class=\"kw1\">in<\/span> list_of_dirs:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">isfile<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> element + <span class=\"st0\">&quot; is a file&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">else<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> <span class=\"st0\">&quot;Processing : &quot;<\/span> + element<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> document <span class=\"kw1\">in<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">listdir<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> document<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; input_file <span class=\"sy0\">=<\/span> pyPdf.<span class=\"me1\">PdfFileReader<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw2\">file<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;rb&quot;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; words <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;big data&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; corruptfile <span class=\"sy0\">=<\/span> <span class=\"kw2\">False<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; numpages <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getNumPages<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; corruptfile <span class=\"sy0\">=<\/span> <span class=\"kw2\">False<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> <span class=\"st0\">&quot;PARSE ERROR : document &quot;<\/span> + document + <span class=\"st0\">&quot; is corrupt&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; corruptfile <span class=\"sy0\">=<\/span> <span class=\"kw2\">True<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> <span class=\"kw1\">not<\/span> corruptfile:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> i <span class=\"kw1\">in<\/span> <span class=\"kw2\">range<\/span><span class=\"br0\">&#40;<\/span><span class=\"nu0\">0<\/span><span class=\"sy0\">,<\/span> input_file.<span class=\"me1\">getNumPages<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getPage<\/span><span class=\"br0\">&#40;<\/span>i<span class=\"br0\">&#41;<\/span>.<span class=\"me1\">extractText<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> <span class=\"st0\">&quot;PARSE ERROR : page : &quot;<\/span> + <span class=\"kw2\">str<\/span><span class=\"br0\">&#40;<\/span>i+<span class=\"nu0\">1<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;&quot;<\/span><br \/>\n<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> text.<span class=\"me1\">lower<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> word <span class=\"kw1\">in<\/span> wordlist:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> word <span class=\"kw1\">in<\/span> text:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; words<span class=\"br0\">&#91;<\/span>word<span class=\"br0\">&#93;<\/span> <span class=\"sy0\">=<\/span> words<span class=\"br0\">&#91;<\/span>word<span class=\"br0\">&#93;<\/span> + <span class=\"st0\">&quot; - &quot;<\/span> + <span class=\"kw2\">str<\/span><span class=\"br0\">&#40;<\/span>i+<span class=\"nu0\">1<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; title <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getDocumentInfo<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span>.<span class=\"me1\">title<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; title <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;PARSE ERROR&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; author <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getDocumentInfo<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span>.<span class=\"me1\">author<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; author <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;PARSE ERROR&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> <span class=\"kw1\">not<\/span> corruptfile :<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;folder&quot;<\/span> : element<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span> : document<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span> : title<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span> : author<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span> : <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">getctime<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;science&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"br0\">&#93;<\/span> <span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;research&quot;<\/span><span class=\"br0\">&#93;<\/span> <span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">else<\/span> : <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;folder&quot;<\/span> : element<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span> : document<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span> : <span class=\"st0\">&quot;CORRUPT FILE&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span> : <span class=\"st0\">&quot;CORRUPT FILE&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span> : <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">getctime<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;science&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"br0\">&#93;<\/span> &nbsp;<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;research&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; writer.<span class=\"me1\">writerow<\/span><span class=\"br0\">&#40;<\/span>line<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; f.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><\/div><\/div>\n<p>Le code que j&#8217;ai finalement utilis\u00e9 (avec PDFMiner) :<\/p>\n<p>&nbsp;<\/p>\n<div class=\"codecolorer-container python default\" style=\"overflow:auto;white-space:nowrap;width:435px;height:300px;\"><div class=\"python codecolorer\"><span class=\"co1\">#!\/usr\/bin\/env python<\/span><br \/>\n<span class=\"co1\"># -*- coding: utf-8 -*-<\/span><br \/>\n<br \/>\n<span class=\"kw1\">import<\/span> unicodecsv <span class=\"kw1\">as<\/span> <span class=\"kw3\">csv<\/span><br \/>\n<span class=\"kw1\">import<\/span> <span class=\"kw3\">os<\/span><span class=\"sy0\">,<\/span> <span class=\"kw3\">sys<\/span><br \/>\n<br \/>\n<span class=\"kw1\">from<\/span> <span class=\"kw3\">datetime<\/span> <span class=\"kw1\">import<\/span> <span class=\"kw3\">datetime<\/span><br \/>\n<br \/>\n<span class=\"kw1\">from<\/span> pdfminer.<span class=\"me1\">pdfinterp<\/span> <span class=\"kw1\">import<\/span> PDFResourceManager<span class=\"sy0\">,<\/span> PDFPageInterpreter<br \/>\n<span class=\"kw1\">from<\/span> pdfminer.<span class=\"me1\">converter<\/span> <span class=\"kw1\">import<\/span> TextConverter<br \/>\n<span class=\"kw1\">from<\/span> pdfminer.<span class=\"me1\">layout<\/span> <span class=\"kw1\">import<\/span> LAParams<br \/>\n<span class=\"kw1\">from<\/span> pdfminer.<span class=\"me1\">pdfpage<\/span> <span class=\"kw1\">import<\/span> PDFPage<br \/>\n<span class=\"kw1\">from<\/span> <span class=\"kw3\">cStringIO<\/span> <span class=\"kw1\">import<\/span> <span class=\"kw3\">StringIO<\/span><br \/>\n<br \/>\n<span class=\"kw1\">import<\/span> pyPdf<br \/>\n<br \/>\n<span class=\"co1\">#Le d\u00e9veloppement de ce script a \u00e9t\u00e9 grandement aid\u00e9 par : https:\/\/stackoverflow.com\/questions\/26494211\/extracting-text-from-a-pdf-file-using-pdfminer-in-python<\/span><br \/>\n<span class=\"co1\">#Et notamment par la solution propos\u00e9e par l'utilisateur : https:\/\/stackoverflow.com\/users\/2930045\/duckpuncher<\/span><br \/>\n<br \/>\n<br \/>\n<span class=\"kw1\">if<\/span> __name__ <span class=\"sy0\">==<\/span> <span class=\"st0\">&quot;__main__&quot;<\/span>:<br \/>\n&nbsp; &nbsp; <br \/>\n<br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; list_of_dirs <span class=\"sy0\">=<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">listdir<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; headers <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;folder&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; wordlist <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; lines <span class=\"sy0\">=<\/span> <span class=\"br0\">&#91;<\/span><span class=\"br0\">&#93;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; f <span class=\"sy0\">=<\/span> <span class=\"kw2\">open<\/span><span class=\"br0\">&#40;<\/span><span class=\"st0\">&quot;output_pdfminer.csv&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;w&quot;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; writer <span class=\"sy0\">=<\/span> <span class=\"kw3\">csv<\/span>.<span class=\"me1\">DictWriter<\/span><span class=\"br0\">&#40;<\/span>f<span class=\"sy0\">,<\/span> fieldnames<span class=\"sy0\">=<\/span>headers<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; writer.<span class=\"me1\">writeheader<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; <span class=\"kw1\">for<\/span> element <span class=\"kw1\">in<\/span> list_of_dirs:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">isfile<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> element + <span class=\"st0\">&quot; is a file&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">else<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> <span class=\"st0\">&quot;Processing : &quot;<\/span> + element<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> document <span class=\"kw1\">in<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">listdir<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> document<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; path <span class=\"sy0\">=<\/span> <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"co1\">#Begin initialising PDFMiner<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; rsrcmgr <span class=\"sy0\">=<\/span> PDFResourceManager<span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; retstr <span class=\"sy0\">=<\/span> <span class=\"kw3\">StringIO<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; codec <span class=\"sy0\">=<\/span> <span class=\"st0\">'utf-8'<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; laparams <span class=\"sy0\">=<\/span> LAParams<span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; device <span class=\"sy0\">=<\/span> TextConverter<span class=\"br0\">&#40;<\/span>rsrcmgr<span class=\"sy0\">,<\/span> retstr<span class=\"sy0\">,<\/span> codec<span class=\"sy0\">=<\/span>codec<span class=\"sy0\">,<\/span> laparams<span class=\"sy0\">=<\/span>laparams<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; fp <span class=\"sy0\">=<\/span> <span class=\"kw2\">file<\/span><span class=\"br0\">&#40;<\/span>path<span class=\"sy0\">,<\/span> <span class=\"st0\">'rb'<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; interpreter <span class=\"sy0\">=<\/span> PDFPageInterpreter<span class=\"br0\">&#40;<\/span>rsrcmgr<span class=\"sy0\">,<\/span> device<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; password <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; maxpages <span class=\"sy0\">=<\/span> <span class=\"nu0\">0<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; caching <span class=\"sy0\">=<\/span> <span class=\"kw2\">False<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pagenos<span class=\"sy0\">=<\/span><span class=\"kw2\">set<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"co1\">#End init PDFMiner<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;&quot;<\/span> &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text2 <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; words <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;big data&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : <span class=\"st0\">&quot;&quot;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; i <span class=\"sy0\">=<\/span> <span class=\"nu0\">1<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> page <span class=\"kw1\">in<\/span> PDFPage.<span class=\"me1\">get_pages<\/span><span class=\"br0\">&#40;<\/span>fp<span class=\"sy0\">,<\/span> pagenos<span class=\"sy0\">,<\/span> maxpages<span class=\"sy0\">=<\/span>maxpages<span class=\"sy0\">,<\/span> password<span class=\"sy0\">=<\/span>password<span class=\"sy0\">,<\/span> check_extractable<span class=\"sy0\">=<\/span><span class=\"kw2\">True<\/span><span class=\"sy0\">,<\/span> caching<span class=\"sy0\">=<\/span>caching<span class=\"br0\">&#41;<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; interpreter.<span class=\"me1\">process_page<\/span><span class=\"br0\">&#40;<\/span>page<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> retstr.<span class=\"me1\">getvalue<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> text.<span class=\"me1\">lower<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">for<\/span> word <span class=\"kw1\">in<\/span> wordlist:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> word <span class=\"kw1\">in<\/span> text:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; words<span class=\"br0\">&#91;<\/span>word<span class=\"br0\">&#93;<\/span> <span class=\"sy0\">=<\/span> words<span class=\"br0\">&#91;<\/span>word<span class=\"br0\">&#93;<\/span> + <span class=\"st0\">&quot; - &quot;<\/span> + <span class=\"kw2\">str<\/span><span class=\"br0\">&#40;<\/span>i<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; i <span class=\"sy0\">=<\/span> i + <span class=\"nu0\">1<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; text <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; retstr.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; retstr <span class=\"sy0\">=<\/span> <span class=\"kw3\">StringIO<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; device <span class=\"sy0\">=<\/span> TextConverter<span class=\"br0\">&#40;<\/span>rsrcmgr<span class=\"sy0\">,<\/span> retstr<span class=\"sy0\">,<\/span> codec<span class=\"sy0\">=<\/span>codec<span class=\"sy0\">,<\/span> laparams<span class=\"sy0\">=<\/span>laparams<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; interpreter <span class=\"sy0\">=<\/span> PDFPageInterpreter<span class=\"br0\">&#40;<\/span>rsrcmgr<span class=\"sy0\">,<\/span> device<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"br0\">&#93;<\/span> <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;PARSE ERROR&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; fp.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; device.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; retstr.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"co1\">#Some of the stuff is going to be parsed through PyPDF<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; input_file <span class=\"sy0\">=<\/span> pyPdf.<span class=\"me1\">PdfFileReader<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw2\">file<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;rb&quot;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; numpages <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getNumPages<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; corruptfile <span class=\"sy0\">=<\/span> <span class=\"kw2\">False<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> <span class=\"st0\">&quot;PARSE ERROR : document &quot;<\/span> + document + <span class=\"st0\">&quot; is corrupt&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; corruptfile <span class=\"sy0\">=<\/span> <span class=\"kw2\">True<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; title <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getDocumentInfo<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span>.<span class=\"me1\">title<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; title <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;PARSE ERROR&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">try<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; author <span class=\"sy0\">=<\/span> input_file.<span class=\"me1\">getDocumentInfo<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span>.<span class=\"me1\">author<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">except<\/span>:<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; author <span class=\"sy0\">=<\/span> <span class=\"st0\">&quot;PARSE ERROR&quot;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; fdate <span class=\"sy0\">=<\/span> <span class=\"kw3\">datetime<\/span>.<span class=\"me1\">fromtimestamp<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">getctime<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span>.<span class=\"me1\">strftime<\/span><span class=\"br0\">&#40;<\/span><span class=\"st0\">'%Y-%m-%d %H:%M:%S'<\/span><span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">if<\/span> <span class=\"kw1\">not<\/span> corruptfile :<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;folder&quot;<\/span> : element<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span> : document<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span> : title<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span> : author<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span> : fdate<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;science&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"br0\">&#93;<\/span> <span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;research&quot;<\/span><span class=\"br0\">&#93;<\/span> <span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">else<\/span> : <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line <span class=\"sy0\">=<\/span> <span class=\"br0\">&#123;<\/span><span class=\"st0\">&quot;folder&quot;<\/span> : element<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;file&quot;<\/span> : document<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;title&quot;<\/span> : <span class=\"st0\">&quot;CORRUPT FILE&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;author&quot;<\/span> : <span class=\"st0\">&quot;CORRUPT FILE&quot;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;date&quot;<\/span> : <span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">getctime<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">path<\/span>.<span class=\"me1\">join<\/span><span class=\"br0\">&#40;<\/span><span class=\"kw3\">os<\/span>.<span class=\"me1\">getcwd<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> element<span class=\"sy0\">,<\/span> document<span class=\"br0\">&#41;<\/span><span class=\"br0\">&#41;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;big data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;big data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;scientific&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;scientific&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;science&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;science&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data mining&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data mining&quot;<\/span><span class=\"br0\">&#93;<\/span> &nbsp;<span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;research&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;research&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"sy0\">,<\/span> <span class=\"st0\">&quot;data&quot;<\/span> : words<span class=\"br0\">&#91;<\/span><span class=\"st0\">&quot;data&quot;<\/span><span class=\"br0\">&#93;<\/span><span class=\"br0\">&#125;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"kw1\">print<\/span> line<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; writer.<span class=\"me1\">writerow<\/span><span class=\"br0\">&#40;<\/span>line<span class=\"br0\">&#41;<\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; f.<span class=\"me1\">close<\/span><span class=\"br0\">&#40;<\/span><span class=\"br0\">&#41;<\/span><\/div><\/div>\n<div class=\"speaker-mute footnotes_reference_container\" > <div class=\"footnote_container_prepare\" ><p ><span class=\"footnote_reference_container_label\" onclick=\"footnote_expand_reference_container_299();\" ><\/span ><span class=\"footnote_reference_container_collapse_button\" style=\"display: none;\" onclick=\"footnote_expand_collapse_reference_container_299();\" >[<a id=\"footnote_reference_container_collapse_button_299\" >+<\/a >]<\/span ><\/p ><\/div > <div id=\"footnote_references_container_299\" style=\"\" > <table class=\"footnote-reference-container\" > <tbody> \r\n\r\n<tr> <td class=\"footnote_plugin_index pointer\"  onclick=\"footnote_moveToAnchor_299('footnote_plugin_tooltip_299_1');\" ><a id=\"footnote_plugin_reference_299_1\" class=\"footnote_backlink\"><span class=\"footnote_index_arrow\">&#8593;<\/span>1<\/a><\/td> <td class=\"footnote_plugin_text\" >Une voire une cat\u00e9gorie d&#8217;articles sur ce sujet devrait \u00eatre d\u00e9di\u00e9 bient\u00f4t sur mon blog. En attendant, vous pouvez aller voir <a href=\"http:\/\/www.costech.utc.fr\/spip.php?article121\">le compte-rendu d&#8217;une journ\u00e9e d&#8217;\u00e9tude<\/a> organis\u00e9e il y a environ un an \u00e0 l&#8217;Institut des sciences de la communication<\/td><\/tr>\r\n\r\n <\/tbody> <\/table> <\/div><\/div><script type=\"text\/javascript\"> function footnote_expand_reference_container_299() { jQuery('#footnote_references_container_299').show(); jQuery('#footnote_reference_container_collapse_button_299').text('\u2212'); } function footnote_collapse_reference_container_299() { jQuery('#footnote_references_container_299').hide(); jQuery('#footnote_reference_container_collapse_button_299').text('+'); } function footnote_expand_collapse_reference_container_299() { if (jQuery('#footnote_references_container_299').is(':hidden')) { footnote_expand_reference_container_299(); } else { footnote_collapse_reference_container_299(); } } function footnote_moveToAnchor_299(p_str_TargetID) { footnote_expand_reference_container_299(); var l_obj_Target = jQuery('#' + p_str_TargetID); if (l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight * 0.20 }, 380);\/*duration*\/ } }<\/script>","protected":false},"excerpt":{"rendered":"<p>Ce billet de blog s&#8217;inscrit dans la s\u00e9rie \u00ab Voyage au pays de la grosse donn\u00e9e \u00bb qui cherche \u00e0 voir ce qu&#8217;il est possible de faire en sciences sociales avec du traitement automatique de texte et du langage sur&#8230;<\/p>\n","protected":false},"author":1,"featured_media":300,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[7,8,9,34,19,35],"class_list":["post-299","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-voyages-au-pays-de-la-grosse-donnee","tag-analyse-textuelle","tag-big-data","tag-methodologie","tag-pdf","tag-protection-des-donnees","tag-python"],"_links":{"self":[{"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/posts\/299","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/comments?post=299"}],"version-history":[{"count":9,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/posts\/299\/revisions"}],"predecessor-version":[{"id":320,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/posts\/299\/revisions\/320"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/media\/300"}],"wp:attachment":[{"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/media?parent=299"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/categories?post=299"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.julienrossi.com\/blog\/wp-json\/wp\/v2\/tags?post=299"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}