Monitorings of Russian disinformation topics
- Слуги стають БПП, бацила націоналізму та “соросята” оточили ЗеленськогоOctober, 14—27 2019
- "Neo-Nazis are preparing the Third Maidan", "Zelenskyi has no real power"September, 30 — October, 13 2019
- Militant Ukrainian authorities, “a bunch of no-names” in the Cabinet of Ministers and the return of YanukovychSeptember, 23—29 2019
- Zelensky is not worthy of a meeting with Putin. Poroshenko will set his house to fire as GontarevaSeptember, 16—22 2019
- Forcing to peace and discrediting the land marketSeptember, 9—15 2019
- Release of hostages, “Zelenskyi corrects Poroshenko's mistakes”September, 2—8 2019
- "Too liberal" Cabinet of Ministers and the threat of putschAugust, 26 — September, 1 2019
- "Bandera Meeting" by Netanyahu, justification of BuzhanskyAugust, 19—25 2019
- Хода РПЦ в Україні, і знову МедведчукJuly, 22—28 2019 року
- Medvedchuk and ShariyJuly, 8—21 2019
- More MedvedchukJune, 24—30 2019
- Poroshenko and Election Circus Are to BlameJune, 17—23 2019
- All attention to the electionsJune, 10—16 2019
- Zelensky can become "bloody"May, 27 — June, 2 2019
- Критика Зеленського, 9 травня, "Одеська трагедія"May, 1—15 2019
Data collection and preprocessing
We were collecting news from a set of Ukrainian sites dominated by clickbait sites (sites of "junk news"). Data was downloaded from sites' RSS feeds or links on their Facebook pages. For each piece of news, we uploaded the date of publication, link, headline, and full text. Next, we defined the language for each text and selected only those that were in Russian. The choice of language was due to an observation that most disinformation is written in Russian. Each text was prepared for analysis: tokenized (divided into language units - words and punctuation marks), lemmatized for topic modeling (words converted to normal form, infinitive), and a vocabulary of words in the news data array was compiled.
All the news was filtered to discard those unrelated to social and political life in Ukraine. Texts about sports, weather, starry life, unrelated international news were filtered out. There were about half of such materials.
Detecting manipulative news
Each news item was evaluated by an improved version of the manipulative news classifier: the algorithm was further trained on new data to improve accuracy (previous version of the classifier). It estimates the likelihood that the news contains emotional manipulation and/or false argumentation. According to news from clickbait sites, sites from the occupied territories and publications with the anti-Ukrainian position, the classifier finds 62% of the materials containing at least one type of manipulation, while incorrectly marks as manipulative 6% of the materials. That is, the algorithm rather misses the manipulation than falsely marks it.
For topic modeling, we selected only news from websites where manipulative news classifier found more than 10% of all news (excluding entertainment, sports, international) contained manipulations.
The following sites were monitored: 112.ua, akcenty.com.ua, alternatio.org, anna-news.info, antifashist.com, antikor.com.ua, argumentua.com, argumentua.life, bbc-ccnn, begemot.media, bessarabiainform.com, beztabu.net, bin.ua, censoru.net, comitet.su, crisis.in.ua, dneprcity.net, donbass.ua, e-news.pro, e-news.su, elise.com.ua, expres.life, for-ua.life, from-ua.com, glavcom.life, glavk.info, glavred.life, golos.ua, hyser.com.ua, inform-ua.info, ipukr.com, ivasi.news, khersonline.net, kievpravda.com, kompromat1.news, kordon.org.ua, merezha.co, metayogg.com, mir-lug.info, most-dnepr.info, nahnews.org, naspravdi.info, nation-news.ru, newnews.in.ua, newsland.com, newsmir.info, newsonline24.com.ua, nk.org.ua, odessamedia.net, odnarodyna.org, onpress.info, osp-ua.info, person.org.ua, pl.com.ua, politeka.net, politica.com.ua, politnavigator.net, prioritet.org, proua.com.ua, prportal.com.ua, replyua.net, rnbee.com.ua, rupor.info, rusnext.ru, rusvesna.su, skelet-info.org, sobytiya.info, spichka.news, spzh.news, t.ks.ua, taras-ua.com, timer-odessa.net, u-news.com.ua, ua-banker.com.ua, ua24ua.net, uazmi.org, ukr.life, ukraina.ru, ukrainianwall.com, ukranews.life, verhovenstvo.com, versii.com, voskhodinfo.su, vybor.ua, укроп.org, znaj.ua, zrada.today
Detection of topics
Selected manipulative news, 3,000 pieces per week on average, was broken down into topics of the week by automatic topic modeling (NMF). We edited the resulting news clusters manually: similar topics combined, irrelevant or overly general clusters discarded, topics that did not follow Russian disinformation deleted. Topics were identified automatically, so a small part of the news may be unrelated to the topic.
Each subtopic is illustrated by a selection headlines that the classifier has identified as manipulative with high confidence.
For monitoring purposes, we have integrated weekly topics into subtopics that are relevant during the monitoring period. The subtopics are grouped into meta-topics for generalization.