The Turkish government has been actively pursuing the prosecution of the participants of the Gülen movement in what it calls
“the Fetullahist Terrorist Organization/Parallel State Structure (FETÖ/PDY)”. To this end, the Turkey’s National Intelligence Organization (Millî İstihbarat Teşkilatı or MİT in Turkish) has investigated the relation of a publicly available smart phone messaging application called ByLock to “FETÖ/PDY”, which is alleged to have been used during the failed coup attempt in Turkey on July 15, 2016.
The MİT is reported to have identified 215,092 users of the ByLock. Of which approximately 75,000 were detained. In an attempt to link a user of ByLock to a real person, the MİT has written a report on its findings which concluded that “ByLock has been offered to the exclusive use of the ‘FTÖ/PDY’ members”.
However, the investigation performed by Fox-IT contradicts the key findings of the MİT. Fox-IT also discovered inconsistencies in the MİT report that indicated manipulation of results and/or screenshots by MİT. What is more, Fox-IT found that the MİT investigation is fundamentally flawed due to the contradictory and baseless findings, lack of objectivity and lack of transparency.
Overall, Fox-IT concluded that the quality of the MİT report is very low, especially when it was weighed against the legal consequences of the conclusions which is the detention of 75,000 Turkish citizens.
This blog contains the conclusions of Fox-IT’s expert witness report. You can also download the full report of Fox-IT and the technical MİT report. The translated version of the MIT report will be made available later.
Reports
Fox-IT’s Expert Witness Report can be downloaded here.
md5: 3dac076d4e0d9d8984533bc04be336a7
sha1: 450ba8665d3a508d569bbc7ec761e9793e42d9c6
sha256: 7dbc402b8e03c311245814fb74dff699330d459f2067ecd574974538086caa7e
MIT’s Technical Report can be downloaded here.
md5: a9f18e08db62ef1c1f71101d37589157
sha1: 68a86e7b8b78c9d1493cb63318dba1f09cc62437
sha256: 17768d91bdae3e78499cb20214bf83abe49948a5d5f41c1aa35a1d1561dd0e62
Conclusions
1. What is Fox-IT’s opinion on the investigation methodology used by MIT in the ByLock investigation?
Multiple key findings from the MIT investigation were contradicted by open-source research conducted by Fox-IT and other findings were shown not to be supported by the evidence presented by MIT. Furthermore, the MIT investigation lacks in transparency: evidence and analysis steps were in many cases omitted from the MIT report. Multiple findings (that could be verified) were shown to be incorrect, which leaves the impression that more findings would proof to be incorrect or inaccurate if they could only be verified.
Fox-IT finds the MIT investigation lacking in objectivity, since there is no indication that MIT investigated the alternative scenario: namely that ByLock has not exclusively been offered to members of the alleged FTÖ/PDY. Investigating alternate scenarios is good practice in an investigation. It helps prevent tunnel vision in cases where investigators are biased towards a predefined outcome. Fox-IT’s examination of the MIT investigation suggests that MIT was, in advance, biased towards the stated conclusion and that MIT has not shown the required objectivity and thoroughness in their investigation to counter this bias.
Fox-IT concludes that the MIT investigation as described in the MIT report does not adhere to the forensic principles as outlined in section 3.1 of this report and should therefore not be regarded as a forensic investigation. The investigation is fundamentally flawed due to the contradicted and unfounded findings, lack of objectivity and lack of transparency. As a result, the conclusions of the investigation are questionable. Fox-IT recommends to conduct a forensic investigation of ByLock in a more thorough, objective and transparent manner.
2. How sound is MIT’s identification of individuals that have used the ByLock application?
The MIT report contains very limited information on the identification of individuals. Fox-IT has shown that ByLock user accounts are, on their own, difficult to attribute to an individual: it is easy to impersonate other individuals when registering a ByLock account and MIT is limited to an IP address from the ByLock server log to identify individuals. Attributing this IP address to actual individuals is not straightforward and error-prone; therefore, possibly leading to identification of the wrong individuals as ByLock users.
Fox-IT is unable to assess the soundness of the identification method, since the MIT report does not provide information on this method. The omission of a description of this method is troubling. Any errors in the method will not be discovered and the reader is left to assume MIT does not make mistakes. While transparency is one of the fundamental principles of forensic investigations, this critical part of the investigation is completely opaque.
3. What is the qualification of soundness on MIT’s conclusion regarding the relation between ByLock and the alleged FTÖ/PDY?
The conclusions and findings of the MIT report were examined by Fox-IT. It was shown that the argumentation is seriously flawed and that seven out of nine stated arguments are incorrect or questionable (see conclusion 6.1). The remaining two arguments are, on their own, not sufficient to support MIT’s conclusion. As a result, the conclusion of the MIT report, “ByLock has been offered to the exclusive use of the members of the terrorist organization of FTÖ/PDY”, is not sound.
4. Are there any other issues identified by Fox-IT that are relevant to the ByLock investigation?
Fox-IT encountered inconsistencies in the MIT report that indicate manipulation of results and/or screenshots by MIT. This is very problematic since it is not clear which of the information in the report stems from original data and which information was modified by MIT (and to which end). This raises questions as to what part of the information available to MIT was altered before presentation, why it was altered and what exactly was left out or changed. When presenting information as evidence, transparency is crucial in differentiating between original data (the actual evidence) and data added or modified by the analyst.
Furthermore, Fox-IT finds the MIT report implicit, not well-structured and lacking in essential details. Bad reporting is not merely a formatting issue. Writing an unreadable report that omits essential details reduces the ability for the reader to scrutinize the investigation that lead to the conclusions. When a report is used as a basis for serious legal consequences, the author should be thorough and concise in the report as to leave no questions regarding the investigation.
Fox-IT has read and written many digital investigation reports over the last 15 years. Based on this experience, Fox-IT finds the quality of the MIT report very low, especially when weighed against the consequences of the conclusions.