EOL

Added EOL

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**NOTE**
+Please consider that some errors could be handled only by providing the info.xml file and the files related to the issue (e.g. a file that cannot be decrypted). If the files needed to understand the bug could contain personal data of any kind, DO NOT SEND THEM. Instead, provide samples that can be shared and with a limited size. Thanks.
+
+**Required info (please complete the following information):**
+ - Huawei Kobackup version: 
+ - Host: [Windows / Linux ]
+ - Kobackup script version:
+ - Kobackup output log (use -vvv)
+
+**Additional context**
+Add any other context about the problem here.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.

README.md

@@ -3,11 +3,11 @@ Huawei backup decryptor
 
 _This script is introduced by the blog post at https://blog.digital-forensics.it/2019/07/huawei-backup-decryptor.html._
 
-The `kobackupdec` is a Python3 script aimed to decrypt Huawei *HiSuite* or *KoBackup* (the Android app) backups. When decrypting and uncompressing the archives, it will re-organize the output folders structure trying to _mimic_ the typical Android one. The script will work both on Windows and Linux hosts, provided the PyCryptoDome dependency.
+The `kobackupdec` is a Python3 script aimed to decrypt Huawei *HiSuite* or *KoBackup* (the Android app) backups. When decrypting and uncompressing the archives, it will re-organize the output folders structure trying to _mimic_ the typical Android one. The script will work both on Windows and Linux hosts, provided the PyCryptoDome dependency. Starting from **20100107** the script was rewritten to handle v9 and v10 kobackup backups structures.
 
-## Update 20100107
+## _EOL_
 
-The script was rewritten to handle v9 and v10 kobackup backups structures.
+On 1.1.2021 the script will get its _end of life_ status. It was needed two years ago to overcome issues for some Huawei devices' forensics acquisitions. Now commercial forensics solutions include the very same capabilities, and much more: there are no more reasons to maintain it. We've got messages from guys using this script to manage theirs backups: we do not recommend it, and we did not write it for this reason. Anyhow we're happy some of you did find it useful, and we thank you for the feedback. We shared it to the community, trying to give back something: if someone has any interest in maintaining it, please let us know so we can include a link to the project. 
 
 ## Usage
 
@@ -16,7 +16,7 @@ The script *assumes* that backups are encrypted with a user-provided password. A
 ```
 usage: kobackupdec.py [-h] [-v] password backup_path dest_path
 
-Huawei KoBackup decryptor version 20190729
+Huawei KoBackup decryptor version 20200611
 
 positional arguments:
   password       user password for the backup
@@ -24,8 +24,10 @@ positional arguments:
   dest_path      decrypted backup folder
 
 optional arguments:
-  -h, --help     show this help message and exit
-  -v, --verbose  verbose level, -v to -vvv
+  -h, --help       show this help message and exit
+  -e, --expandtar  expand tar files
+  -w, --writable   do not set RO pemission on decrypted data
+  -v, --verbose    verbose level, -v to -vvv
 ```
 
 - `password`, is the user provided password.

kobackupdec.py

@@ -4,12 +4,21 @@
 # Huawei KoBackup backups decryptor.
 #
 # Version History
+# - 20200705: fixed decrypt_large_package to read input's chunks
+# - 20200611: added 'expandtar' option, to avoid automatic expansion of TARs
+#             added 'writable' option, to allow user RW on decrypted files
+#             large TAR files are not managed in chunk but not expanded
+# - 20200607: merged empty CheckMsg, update folder_to_media_type by @realSnoopy
+# - 20200406: merged pull by @lp4n6, related to files and folders permissions
+# - 20200405: added Python minor version check and note (thanks @lp4n6)
 # - 2020test: rewritten to handle v9 and v10 backups
 # - 20200107: merged pull by @lp4n6, fixed current version
 # - 20191113: fixed double folder creation error
 # - 20190729: first public release
 # - 20190729: first public release
 #
+# Note: it needs Python version >= 3.7
+#
 # Released under MIT License
 #
 # Copyright (c) 2019 Francesco "dfirfpi" Picasso, Reality Net System Solutions
@@ -52,11 +61,13 @@ from Crypto.Hash import HMAC
 from Crypto.Protocol.KDF import PBKDF2
 from Crypto.Util import Counter
 
-VERSION = '2020test'
+VERSION = '20200705'
 
 # Disabling check on doc strings and naming convention.
 # pylint: disable=C0111,C0103
 
+MAX_FILE_SIZE = 536870912 # Files larger than that needs to be 'chuncked'.
+
 # --- DecryptMaterial ---------------------------------------------------------
 
 class DecryptMaterial:
@@ -159,6 +170,7 @@ class Decryptor:
 
     count = 5000
     dklen = 32
+    chunk_size = 1024*1024*64
 
     def __init__(self, password):
         '''Initialize the object by setting a password.'''
@@ -262,21 +274,28 @@ class Decryptor:
         logging.debug('SHA256(BKEY)[%s] = %s', len(self._bkey_sha256),
                       binascii.hexlify(self._bkey_sha256))
 
-        salt = self._checkMsg[32:]
-        logging.debug('SALT[%s] = %s', len(salt), binascii.hexlify(salt))
+        # [TBR][TODO] This check should be refactored.
+        if self._checkMsg:
+            salt = self._checkMsg[32:]
 
-        res = PBKDF2(self._bkey, salt, Decryptor.dklen, Decryptor.count,
-                     Decryptor.prf, hmac_hash_module=None)
-        logging.debug('KEY check expected = %s',
-                      binascii.hexlify(self._checkMsg[:32]))
-        logging.debug('RESULT = %s', binascii.hexlify(res))
+            logging.debug('SALT[%s] = %s', len(salt), binascii.hexlify(salt))
 
-        if res == self._checkMsg[:32]:
-            logging.info('OK, backup key is correct!')
-            self._good = True
+            res = PBKDF2(self._bkey, salt, Decryptor.dklen, Decryptor.count,
+                         Decryptor.prf, hmac_hash_module=None)
+            logging.debug('KEY check expected = %s',
+                          binascii.hexlify(self._checkMsg[:32]))
+            logging.debug('RESULT = %s', binascii.hexlify(res))
+
+            if res == self._checkMsg[:32]:
+                logging.info('OK, backup key is correct!')
+                self._good = True
+            else:
+                logging.error('KO, backup key is wrong!')
+                self._good = False
         else:
-            logging.error('KO, backup key is wrong!')
-            self._good = False
+            logging.warning('Empty CheckMsg! Cannot check backup password!')
+            logging.warning('Assuming the provided password is correct...')
+            self._good = True
 
     def decrypt_package(self, dec_material, data):
         if not self._good:
@@ -298,6 +317,31 @@ class Decryptor:
         decryptor = AES.new(key, mode=AES.MODE_CTR, counter=counter_obj)
         return decryptor.decrypt(data)
 
+    def decrypt_large_package(self, dec_material, entry):
+        if not self._good:
+            logging.warning('well, it is hard to decrypt with a wrong key.')
+
+        if not dec_material.encMsgV3:
+            logging.error('cannot decrypt with an empty encMsgV3!')
+            return None
+
+        salt = dec_material.encMsgV3[:32]
+        counter_iv = dec_material.encMsgV3[32:]
+
+        key = PBKDF2(self._bkey, salt, Decryptor.dklen, Decryptor.count,
+                     Decryptor.prf, hmac_hash_module=None)
+
+        counter_obj = Counter.new(128, initial_value=int.from_bytes(
+            counter_iv, byteorder='big'), little_endian=False)
+
+        decryptor = AES.new(key, mode=AES.MODE_CTR, counter=counter_obj)
+        data_len = entry.stat().st_size
+        with open(entry, 'rb') as entry_fd:
+            for x in range(0, data_len, self.chunk_size):
+                logging.debug('decrypting chunk %d of %s', x, entry)
+                data = entry_fd.read(self.chunk_size)
+                yield decryptor.decrypt(data)
+
     def decrypt_file(self, dec_material, data):
         if not self._good:
             logging.warning('well, it is hard to decrypt with a wrong key.')
@@ -648,13 +692,26 @@ def decrypt_entry(decrypt_info, entry, type_info, search=False):
         logging.warning('entry %s has no decrypt material!', skey)
     return cleartext
 
+# --- decrypt_large_entry -----------------------------------------------------
+
+def decrypt_large_entry(decrypt_info, entry, type_info, search=False):
+    skey = entry.stem
+    decrypt_material = decrypt_info.get_decrypt_material(skey, type_info,
+                                                         search)
+    if decrypt_material:
+        for x in decrypt_info.decryptor.decrypt_large_package(
+                decrypt_material, entry):
+            yield x
+    else:
+        logging.warning('entry %s has no decrypt material!', skey)
+
 # --- decrypt_files_in_root ---------------------------------------------------
 
-def decrypt_files_in_root(decrypt_info, path_in, path_out):
+def decrypt_files_in_root(decrypt_info, path_in, path_out, expandtar):
 
     data_apk_dir = path_out.absolute().joinpath('data/app')
     data_app_dir = path_out.absolute().joinpath('data/data')
-    #data_app_dir.mkdir(parents=True, exist_ok=True)
+    #data_app_dir.mkdir(0o755, parents=True, exist_ok=True)
     data_unk_dir = path_out.absolute().joinpath('unknown')
 
     for entry in path_in.glob('*'):
@@ -670,7 +727,7 @@ def decrypt_files_in_root(decrypt_info, path_in, path_out):
 
         if extension == '.apk':
             dest_file = data_apk_dir.joinpath(entry.name + '-1')
-            dest_file.mkdir(parents=True, exist_ok=True)
+            dest_file.mkdir(0o755, parents=True, exist_ok=True)
             dest_file = dest_file.joinpath('base.apk')
             dest_file.write_bytes(entry.read_bytes())
 
@@ -680,38 +737,60 @@ def decrypt_files_in_root(decrypt_info, path_in, path_out):
                                       search=True)
             if cleartext:
                 dest_file = data_app_dir.joinpath(entry.name)
-                dest_file.parent.mkdir(parents=True, exist_ok=True)
+                dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
                 dest_file.write_bytes(cleartext)
             else:
                 logging.warning('unable to decrypt entry %s', entry.name)
 
-        elif extension == '.tar':
+        elif extension == '.tar' and entry.stat().st_size < MAX_FILE_SIZE:
             cleartext = decrypt_entry(decrypt_info, entry,
                                       DecryptInfo.info_type.FILE)
-            if cleartext:
+            if cleartext and expandtar:
                 with tarfile.open(fileobj=io.BytesIO(cleartext)) as tar_data:
                     if os.name == 'nt':
                         tar_extract_win(tar_data, data_app_dir)
                     else:
                         tar_data.extractall(path=data_app_dir)
+            elif cleartext:
+                logging.info('Not expanding TAR file %s', entry.name)
+                dest_file = data_app_dir.joinpath(entry.name)
+                dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
+                dest_file.write_bytes(cleartext)
             else:
                 logging.warning('unable to decrypt entry %s', entry.name)
 
+        elif extension == '.tar' and entry.stat().st_size >= MAX_FILE_SIZE:
+            logging.info('Decrypting LARGE entry %s', entry.name)
+            logging.info('TAR will not be expanded')
+            dest_file = data_app_dir.joinpath(entry.name)
+            dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
+            with open(dest_file, 'wb') as fd:
+                for x in decrypt_large_entry(decrypt_info, entry,
+                                             DecryptInfo.info_type.FILE):
+                    fd.write(x)
+
         else:
             logging.warning('entry %s unmanged, copying it', entry.name)
             dest_file = data_unk_dir.joinpath(entry.name)
-            dest_file.parent.mkdir(parents=True, exist_ok=True)
+            dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
             dest_file.write_bytes(entry.read_bytes())
 
 # --- decrypt_files_in_folder -------------------------------------------------
 
-def decrypt_files_in_folder(decrypt_info, folder, path_out):
+def decrypt_files_in_folder(decrypt_info, folder, path_out, expandtar):
 
-    folder_to_media_type = {'movies': 'video', 'pictures': 'photo'}
+    folder_to_media_type = {'movies': 'video', 'pictures': 'photo',
+                            'audios': 'audio', }
 
     media_out_dir = path_out.absolute().joinpath('storage')
     media_unk_dir = path_out.absolute().joinpath('unknown')
 
+    # Dirty 'hack' to see if an XML file is inside the folder with IVs
+    # needed to decrypt .enc files... Not tested for side effects.
+    xml_files = folder.glob('*.xml')
+    for entry in xml_files:
+        parse_generic_xml(entry, decrypt_info)
+
     for entry in folder.glob('**/*'):
         if entry.is_dir():
             continue
@@ -732,7 +811,7 @@ def decrypt_files_in_folder(decrypt_info, folder, path_out):
             if cleartext and decrypt_material:
                 tmp_path = decrypt_material.path.lstrip('/').lstrip('\\')
                 dest_file = path_out.joinpath(tmp_path)
-                dest_file.parent.mkdir(parents=True, exist_ok=True)
+                dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
                 dest_file.write_bytes(cleartext)
                 continue
 
@@ -749,7 +828,7 @@ def decrypt_files_in_folder(decrypt_info, folder, path_out):
                 decrypt_material, entry.read_bytes())
             if cleartext:
                 dest_file = media_out_dir.joinpath(entry.relative_to(folder))
-                dest_file.parent.mkdir(parents=True, exist_ok=True)
+                dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
                 dest_file.write_bytes(cleartext)
                 continue
 
@@ -761,8 +840,8 @@ def decrypt_files_in_folder(decrypt_info, folder, path_out):
                 decrypt_material, entry.read_bytes())
             if cleartext:
                 dest_file = media_out_dir.joinpath(entry.relative_to(folder))
-                dest_file.parent.mkdir(parents=True, exist_ok=True)
-                if entry.suffix.lower() == '.tar':
+                dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
+                if entry.suffix.lower() == '.tar' and expandtar:
                     with tarfile.open(fileobj=io.BytesIO(cleartext)) as tdata:
                         if os.name == 'nt':
                             tar_extract_win(tdata, dest_file.parent)
@@ -772,20 +851,20 @@ def decrypt_files_in_folder(decrypt_info, folder, path_out):
                 if dest_file.exists():
                     new_name = str(folder.name) + '_' + str(dest_file.name)
                     dest_file = dest_file.parent.joinpath(new_name)
-                    dest_file.parent.mkdir(parents=True, exist_ok=True)
+                    dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
                 dest_file.write_bytes(cleartext)
                 continue
 
         if cleartext is None:
             logging.warning('decrypting [%s] failed, copying it', entry.name)
             dest_file = media_unk_dir.joinpath(entry.name)
-            dest_file.parent.mkdir(parents=True, exist_ok=True)
+            dest_file.parent.mkdir(0o755, parents=True, exist_ok=True)
             dest_file.write_bytes(entry.read_bytes())
 
 
 # --- decrypt_backup ----------------------------------------------------------
 
-def decrypt_backup(password, path_in, path_out):
+def decrypt_backup(password, path_in, path_out, expandtar):
 
     decrypt_info = parse_info_xml(path_in.joinpath('info.xml'), password)
     if not decrypt_info:
@@ -798,20 +877,20 @@ def decrypt_backup(password, path_in, path_out):
 
     xml_files = path_in.glob('*.xml')
     for entry in xml_files:
-        if entry.name != 'info.xml':
+        if entry.name != 'info.xml' and not entry.name.startswith('._'):
             parse_generic_xml(entry, decrypt_info)
 
     logging.debug(decrypt_info.dump())
 
-    decrypt_files_in_root(decrypt_info, path_in, path_out)
+    decrypt_files_in_root(decrypt_info, path_in, path_out, expandtar)
 
     for entry in path_in.glob('*'):
         if entry.is_dir():
-            decrypt_files_in_folder(decrypt_info, entry, path_out)
+            decrypt_files_in_folder(decrypt_info, entry, path_out, expandtar)
 
 # --- decrypt_media -----------------------------------------------------------
 
-def decrypt_media(password, path_in, path_out):
+def decrypt_media(password, path_in, path_out, expandtar):
 
     # [TODO][TBR] Should parse media.db sqlite.
 
@@ -833,11 +912,11 @@ def decrypt_media(password, path_in, path_out):
 
     for entry in subfolder.glob('*'):
         if entry.is_dir():
-            decrypt_files_in_folder(decrypt_info, entry, path_out)
+            decrypt_files_in_folder(decrypt_info, entry, path_out, expandtar)
 
 # --- main --------------------------------------------------------------------
 
-def main(password, backup_path_in, dest_path_out):
+def main(password, backup_path_in, dest_path_out, expandtar, writable):
 
     logging.info('searching backup in [%s]', backup_path_in)
 
@@ -859,7 +938,7 @@ def main(password, backup_path_in, dest_path_out):
 
     if files_folder:
         logging.info('got info.xml, going to decrypt backup files')
-        decrypt_backup(password, files_folder, dest_path_out)
+        decrypt_backup(password, files_folder, dest_path_out, expandtar)
 
     media_folder = None
     if backup_path_in.joinpath('media').is_dir():
@@ -869,11 +948,20 @@ def main(password, backup_path_in, dest_path_out):
         logging.info('No media folder found.')
 
     if media_folder:
-        decrypt_media(password, media_folder, dest_path_out)
+        decrypt_media(password, media_folder, dest_path_out, expandtar)
+
+    if writable:
+        logging.info('Not setting read-only on decrypted files')
+    else:
+        logging.info('setting all decrypted files to read-only')
+        for entry in dest_path_out.glob('**/*'):
+            # Set read-only permission if entry is a file.
+            if os.path.isfile(entry):
+                os.chmod(entry, 0o444)
+            # *nix directories require execute permission to read/traverse
+            elif os.path.isdir(entry):
+                os.chmod(entry, 0o555)
 
-    logging.info('setting all decrypted files to read-only')
-    for entry in dest_path_out.glob('**/*'):
-        os.chmod(entry, 0o444)
 
 # --- entry point and parameters checks ---------------------------------------
 
@@ -881,12 +969,18 @@ if __name__ == '__main__':
 
     if sys.version_info[0] < 3:
         sys.exit('Python 3 or a more recent version is required.')
+    elif sys.version_info[1] < 7:
+        sys.exit('Python 3.7 or a more recent version is required.')
 
     description = 'Huawei KoBackup decryptor version {}'.format(VERSION)
     parser = argparse.ArgumentParser(description=description)
     parser.add_argument('password', help='user password for the backup')
     parser.add_argument('backup_path', help='backup folder')
     parser.add_argument('dest_path', help='decrypted backup folder')
+    parser.add_argument('-e', '--expandtar', action='store_true',
+                        help='expand tar files')
+    parser.add_argument('-w', '--writable', action='store_true',
+                        help='do not set RO pemission on decrypted data')
     parser.add_argument('-v', '--verbose', action='count',
                         help='verbose level, -v to -vvv')
     args = parser.parse_args()
@@ -912,6 +1006,8 @@ if __name__ == '__main__':
     dest_path = pathlib.Path(args.dest_path)
     if dest_path.is_dir():
         sys.exit('Destination folder already exists!')
-    dest_path.mkdir(parents=True)
 
-    main(user_password, backup_path, dest_path)
+    # Make directory with read and execute permission (=read and traverse)
+    dest_path.mkdir(0o755, parents=True)
+
+    main(user_password, backup_path, dest_path, args.expandtar, args.writable)

setup.py

@@ -0,0 +1,17 @@
+# Setup file for compiling the python script with cx_Freeze (https://github.com/anthony-tuininga/cx_Freeze)
+
+from cx_Freeze import setup, Executable
+
+executables = [
+    Executable('kobackupdec.py')
+]
+
+setup(name='KoBackupDec',
+# Change build number to the current one
+    version='20200607',
+    description='HiSuite / KoBackup Decryptor',
+    executables=executables
+)
+
+# Compile the python script to an executable with: python setup.py build
+# Build an Windows installation Package with: python setup.py bdist_msi