From: Moritz Warning <moritzwarning@web.de>
Date: Thu, 17 Sep 2020 20:00:47 +0000 (+0200)
Subject: misc/collect.py: introduce variable {base} to fix download_url
X-Git-Tag: v3.0.3~5
X-Git-Url: http://git.openwrt.org/openwrt/feeds.git?p=web%2Ffirmware-selector-openwrt-org.git;a=commitdiff_plain;h=d6a169ef822a9aedef532eceac6a1fced32623c6

misc/collect.py: introduce variable {base} to fix download_url

While scanning a directory for profile.json files of different releases,
the releases might be in different sub directories. But the collect.py
script only takes one download URL template for all overview.json files.

This commit introduces a variable {base} for the download URL parameter
that will be replaced by the sub directory that distinguish the location
of different release from each other.

Example path structure:
* ../tmp/releases/18.06.8/targets
* ../tmp/snapshot/targets

collect.py call:
```
./misc/collect.py scan "https://download.openwrt.org/{base}/{target}" ../tmp/ www/
```
In www/config.json:
```
versions: {'18.06.8': 'data/18.06.8/overview.json', 'SNAPSHOT': 'data/SNAPSHOT/overview.json'},
```

In www/data/18.06.8/overview.json:
```
"download_url": "https://download.openwrt.org/releases/18.06.8/targets/{target}",
```

In www/data/SNAPSHOT/overview.json:
```
"download_url": "https://download.openwrt.org/snapshots/targets/{target}"
```
---

diff --git a/misc/collect.py b/misc/collect.py
index fbaa4ce..527d27a 100755
--- a/misc/collect.py
+++ b/misc/collect.py
@@ -329,8 +329,30 @@ def scan(args):
                 }
             )
 
+    """
+        Replace {base} variable in download URL with the intersection
+        of all profile.json paths. E.g.:
+        ../tmp/releases/18.06.8/targets => base is releases/18.06.8/targets
+        ../tmp/snapshots/targets => base in snapshots/targets
+    """
+
+    def replace_base(releases, target_release, download_url):
+        if "{base}" in download_url:
+            # release => base path (of profiles.json locations)
+            paths = {}
+            for release, profiles in releases.items():
+                paths[release] = os.path.commonpath(profiles.keys())
+            # base path of all releases
+            release_path_base = os.path.commonpath(paths.values())
+            # get path intersection
+            base = str(paths[target_release])[len(release_path_base) + 1 :]
+            return download_url.replace("{base}", base)
+        else:
+            return download_url
+
     for release, profiles in releases.items():
-        output = merge_profiles(profiles, args.download_url)
+        download_url = replace_base(releases, release, args.download_url)
+        output = merge_profiles(profiles, download_url)
 
         versions[release] = "data/{}/overview.json".format(release)
         os.makedirs("{}/{}".format(data_path, release), exist_ok=True)